Max level shown:
12025-04-30T06:12:51.790ZDEBGomicron-dev: Ensuring seed tarball exists
22025-04-30T06:12:51.821ZINFOomicron-dev: Using existing CRDB seed tarball: `/var/tmp/omicron_tmp/crdb-base-build/f234518892194a05ad8767dc97f01a61a55baf1431942a314ce8b0f72b18ec2e.tar`
32025-04-30T06:12:51.821ZDEBGomicron-dev (init_with_steps): Running step start_crdb
42025-04-30T06:12:51.822ZDEBGomicron-dev: Starting CRDB
52025-04-30T06:12:52.664ZINFOomicron-dev: cockroach temporary directory: /var/tmp/omicron_tmp/.tmpuKFLrK
62025-04-30T06:12:52.664ZINFOomicron-dev: cockroach: copying from seed tarball (/var/tmp/omicron_tmp/crdb-base-build/f234518892194a05ad8767dc97f01a61a55baf1431942a314ce8b0f72b18ec2e.tar) to storage directory (/var/tmp/omicron_tmp/.tmpuKFLrK/data)
72025-04-30T06:12:52.701ZINFOomicron-dev: cockroach command line: cockroach start-single-node --insecure --http-addr=:0 --store=path=/var/tmp/omicron_tmp/.tmpuKFLrK/data,ballast-size=0 --listen-addr [::1]:0 --listening-url-file /var/tmp/omicron_tmp/.tmpuKFLrK/listen-url
82025-04-30T06:12:52.705ZINFOomicron-dev: cockroach environment: BUILDOMAT_JOB_ID=01JT2JWKGHF3JHFMW0JR13NSA6 BUILDOMAT_TASK_ID=4 CARGO=/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/bin/cargo CARGO_HOME=/home/build/.cargo CARGO_INCREMENTAL=0 CARGO_MANIFEST_DIR=/work/oxidecomputer/omicron/dev-tools/omicron-dev CARGO_PKG_AUTHORS= CARGO_PKG_DESCRIPTION= CARGO_PKG_HOMEPAGE= CARGO_PKG_LICENSE=MPL-2.0 CARGO_PKG_LICENSE_FILE= CARGO_PKG_NAME=omicron-dev CARGO_PKG_REPOSITORY= CARGO_PKG_RUST_VERSION= CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE= CARGO_TERM_COLOR=always CI=true GITHUB_REPOSITORY=oxidecomputer/omicron GITHUB_SHA=531912b3fc900900bc7bd18f5eac1ce1e22cb1cd GOTRACEBACK=crash HOME=/home/build LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-6532f52db341a7cc/out/build/artifacts:/work/oxidecomputer/omicron/target/debug/build/blake3-403ee89fc24db8f8/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-aece66bfb5c8df92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-5fe90ecb72e1690f/out/build:/work/oxidecomputer/omicron/target/debug/build/ring-fb368fe91b2bd145/out:/work/oxidecomputer/omicron/target/debug/build/tofino-ffe28f7f2301d0c2/out:/work/oxidecomputer/omicron/target/debug/build/zstd-sys-a7fb19a1e17b70f6/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib LOGNAME=build NEXTEST=1 NEXTEST_BIN_EXE_omicron-dev=/work/oxidecomputer/omicron/target/debug/omicron-dev NEXTEST_EXECUTION_MODE=process-per-test NEXTEST_LD_LIBRARY_PATH=/work/oxidecomputer/omicron/target/debug/build/aws-lc-sys-6532f52db341a7cc/out/build/artifacts:/work/oxidecomputer/omicron/target/debug/build/blake3-403ee89fc24db8f8/out:/work/oxidecomputer/omicron/target/debug/build/bzip2-sys-aece66bfb5c8df92/out/lib:/work/oxidecomputer/omicron/target/debug/build/libgit2-sys-5fe90ecb72e1690f/out/build:/work/oxidecomputer/omicron/target/debug/build/ring-fb368fe91b2bd145/out:/work/oxidecomputer/omicron/target/debug/build/tofino-ffe28f7f2301d0c2/out:/work/oxidecomputer/omicron/target/debug/build/zstd-sys-a7fb19a1e17b70f6/out:/work/oxidecomputer/omicron/target/debug/deps:/work/oxidecomputer/omicron/target/debug:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib/rustlib/x86_64-unknown-illumos/lib:/home/build/.rustup/toolchains/1.86.0-x86_64-unknown-illumos/lib NEXTEST_PROFILE=ci NEXTEST_RUN_ID=29735cc2-0883-4059-8ca2-1da53f01ede3 NEXTEST_TEST_GLOBAL_SLOT=2 NEXTEST_TEST_GROUP=@global NEXTEST_TEST_GROUP_SLOT=none OUT_DIR=/work/oxidecomputer/omicron/target/debug/build/omicron-dev-cc596a50d8d064c3/out PATH=/work/oxidecomputer/omicron/out/mgd/root/opt/oxide/mgd/bin:/work/oxidecomputer/omicron/out/dendrite-stub/bin:/work/oxidecomputer/omicron/out/clickhouse:/work/oxidecomputer/omicron/out/cockroachdb/bin:/home/build/.cargo/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/ooce/bin:/opt/ooce/sbin PWD=/work/oxidecomputer/omicron/dev-tools/omicron-dev RUSTC_BOOTSTRAP=1 RUSTDOCFLAGS=--document-private-items -D warnings RUSTFLAGS=-D warnings -C link-arg=-R/usr/platform/oxide/lib/amd64 RUSTUP_HOME=/home/build/.rustup RUSTUP_TOOLCHAIN=1.86.0-x86_64-unknown-illumos RUST_BACKTRACE=1 RUST_RECURSION_COUNT=1 SHLVL=2 SSL_CERT_DIR=/usr/ssl/certs SSL_CERT_FILE=/etc/ssl/cacert.pem TMPDIR=/var/tmp/omicron_tmp TZ=UTC USER=build _=/work/oxidecomputer/omicron/target/debug/omicron-dev __NEXTEST_ATTEMPT=1
92025-04-30T06:12:55.156ZINFOomicron-dev: cockroach pid: 17874
102025-04-30T06:12:55.156ZINFOomicron-dev: cockroach listen URL: postgresql://root@[::1]:60242/omicron?sslmode=disable
112025-04-30T06:12:55.169ZDEBGomicron-dev (init_with_steps): Running step start_clickhouse
122025-04-30T06:12:55.169ZDEBGomicron-dev: Starting Clickhouse
132025-04-30T06:12:56.462ZDEBGomicron-dev (init_with_steps): Running step start_gateway_switch0
142025-04-30T06:12:56.462ZDEBGomicron-dev: Starting Management Gateway
152025-04-30T06:12:56.549ZDEBGomicron-dev (init_with_steps): Running step start_dendrite_switch0
162025-04-30T06:12:56.549ZDEBGomicron-dev: Starting Dendrite for switch0
172025-04-30T06:12:56.623ZDEBGomicron-dev (init_with_steps): Running step start_mgd_switch0
182025-04-30T06:12:56.623ZDEBGomicron-dev: Starting mgd for switch0
192025-04-30T06:12:56.846ZDEBGomicron-dev: mgd port is 54778
202025-04-30T06:12:56.847ZDEBGomicron-dev (init_with_steps): Running step record_switch_dns
212025-04-30T06:12:56.854ZDEBGomicron-dev: Recording DNS for the switch zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 switch_location = switch0
222025-04-30T06:12:56.860ZDEBGomicron-dev (init_with_steps): Running step start_internal_dns
232025-04-30T06:12:56.868ZINFOomicron-dev (store): opening sled database path = /var/tmp/omicron_tmp/.tmphx1Wr8
242025-04-30T06:12:57.122ZINFOomicron-dev (store): pruning trees for generations newer than 0
252025-04-30T06:12:57.122ZINFOomicron-dev (store): pruning trees for generations older than 0 keep = 3
262025-04-30T06:12:57.135ZINFOomicron-dev (dns): DNS server bound to address kind = dns local_address = [::1]:48988
272025-04-30T06:12:57.139ZDEBGomicron-dev (http): registered endpoint kind = dns local_addr = [::1]:39475 method = GET path = /config versions = all
282025-04-30T06:12:57.139ZDEBGomicron-dev (http): registered endpoint kind = dns local_addr = [::1]:39475 method = PUT path = /config versions = all
292025-04-30T06:12:57.139ZINFOomicron-dev (http): listening kind = dns local_addr = [::1]:39475
302025-04-30T06:12:57.140ZDEBGomicron-dev (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:39475
312025-04-30T06:12:57.141ZDEBGomicron-dev (init_with_steps): Running step start_external_dns
322025-04-30T06:12:57.141ZINFOomicron-dev (store): opening sled database path = /var/tmp/omicron_tmp/.tmp6aHvHa
332025-04-30T06:12:57.158ZINFOomicron-dev (store): pruning trees for generations newer than 0
342025-04-30T06:12:57.158ZINFOomicron-dev (store): pruning trees for generations older than 0 keep = 3
352025-04-30T06:12:57.158ZINFOomicron-dev (dns): DNS server bound to address kind = dns local_address = [::1]:55326
362025-04-30T06:12:57.159ZDEBGomicron-dev (http): registered endpoint kind = dns local_addr = [::1]:54696 method = GET path = /config versions = all
372025-04-30T06:12:57.159ZDEBGomicron-dev (http): registered endpoint kind = dns local_addr = [::1]:54696 method = PUT path = /config versions = all
382025-04-30T06:12:57.159ZINFOomicron-dev (http): listening kind = dns local_addr = [::1]:54696
392025-04-30T06:12:57.160ZDEBGomicron-dev (http): successfully registered DTrace USDT probes kind = dns local_addr = [::1]:54696
402025-04-30T06:12:57.176ZDEBGomicron-dev (init_with_steps): Running step start_nexus_internal
412025-04-30T06:12:57.176ZDEBGomicron-dev: Starting Nexus (internal API)
422025-04-30T06:12:57.250ZINFOomicron-dev: setting up nexus server
432025-04-30T06:12:57.367ZINFOomicron-dev (ServerContext): registering Oso class class = Action
442025-04-30T06:12:57.367ZINFOomicron-dev (ServerContext): registering Oso class class = AnyActor
452025-04-30T06:12:57.367ZINFOomicron-dev (ServerContext): registering Oso class class = AuthenticatedActor
462025-04-30T06:12:57.367ZINFOomicron-dev (ServerContext): registering Oso class class = BlueprintConfig
472025-04-30T06:12:57.367ZINFOomicron-dev (ServerContext): registering Oso class class = Database
482025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = DnsConfig
492025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = Fleet
502025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = Inventory
512025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = IpPoolList
522025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = ConsoleSessionList
532025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = DeviceAuthRequestList
542025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = SiloCertificateList
552025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = SiloIdentityProviderList
562025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = SiloUserList
572025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = TargetReleaseConfig
582025-04-30T06:12:57.368ZINFOomicron-dev (ServerContext): registering Oso class class = WebhookEventClassList
592025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = Project
602025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = Disk
612025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = Snapshot
622025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = ProjectImage
632025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = AffinityGroup
642025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = AntiAffinityGroup
652025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = Instance
662025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = IpPool
672025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = InstanceNetworkInterface
682025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = Vpc
692025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = VpcRouter
702025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = InternetGateway
712025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = InternetGatewayIpPool
722025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = InternetGatewayIpAddress
732025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = RouterRoute
742025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = VpcSubnet
752025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = FloatingIp
762025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = Image
772025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = SiloImage
782025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = AddressLot
792025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = Blueprint
802025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = LoopbackAddress
812025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = Certificate
822025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = ConsoleSession
832025-04-30T06:12:57.369ZINFOomicron-dev (ServerContext): registering Oso class class = DeviceAuthRequest
842025-04-30T06:12:57.370ZINFOomicron-dev (ServerContext): registering Oso class class = DeviceAccessToken
852025-04-30T06:12:57.370ZINFOomicron-dev (ServerContext): registering Oso class class = PhysicalDisk
862025-04-30T06:12:57.370ZINFOomicron-dev (ServerContext): registering Oso class class = Rack
872025-04-30T06:12:57.370ZINFOomicron-dev (ServerContext): registering Oso class class = RoleBuiltin
882025-04-30T06:12:57.370ZINFOomicron-dev (ServerContext): registering Oso class class = SshKey
892025-04-30T06:12:57.370ZINFOomicron-dev (ServerContext): registering Oso class class = Silo
902025-04-30T06:12:57.370ZINFOomicron-dev (ServerContext): registering Oso class class = SiloUser
912025-04-30T06:12:57.370ZINFOomicron-dev (ServerContext): registering Oso class class = SiloGroup
922025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = SupportBundle
932025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = IdentityProvider
942025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = SamlIdentityProvider
952025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = Sled
962025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = TufRepo
972025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = TufArtifact
982025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = WebhookEvent
992025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = WebhookReceiver
1002025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = WebhookSecret
1012025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = Zpool
1022025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = Service
1032025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): registering Oso class class = UserBuiltin
1042025-04-30T06:12:57.371ZINFOomicron-dev (ServerContext): full Oso configuration config = #\n# Oso configuration for Omicron\n# This file is augmented by generated snippets.\n#\n\n#\n# ACTOR TYPES AND BASIC RULES\n#\n\n# `AnyActor` includes both authenticated and unauthenticated users.\nactor AnyActor {}\n\n# An `AuthenticatedActor` has an identity in the system. All of our operations\n# today require that an actor be authenticated.\nactor AuthenticatedActor {}\n\n# For any resource, `actor` can perform action `action` on it if they're\n# authenticated and their role(s) give them the corresponding permission on that\n# resource.\nallow(actor: AnyActor, action: Action, resource) if\n actor.authenticated and\n has_permission(actor.authn_actor.unwrap(), action.to_perm(), resource);\n\n# Define role relationships\nhas_role(actor: AuthenticatedActor, role: String, resource: Resource)\n\tif resource.has_role(actor, role);\n\n#\n# ROLES AND PERMISSIONS IN THE FLEET/SILO/PROJECT HIERARCHY\n#\n# We define the following permissions for most resources in the system:\n#\n# - "create_child": required to create child resources (of any type)\n#\n# - "list_children": required to list child resources (of all types) of a\n# resource\n#\n# - "modify": required to modify or delete a resource\n#\n# - "read": required to read a resource\n#\n# We define the following predefined roles for only a few high-level resources:\n# the Fleet (see below), Silo, Organization, and Project. The specific roles\n# are oriented around intended use-cases:\n#\n# - "admin": has all permissions on the resource\n#\n# - "collaborator": has "read", "list_children", and "create_child", plus\n# the "admin" role for child resources. The idea is that if you're an\n# Organization Collaborator, you have full control over the Projects within\n# the Organization, but you cannot modify or delete the Organization itself.\n#\n# - "viewer": has "read" and "list_children" on a resource\n#\n# Below the Project level, permissions are granted via roles at the Project\n# level. For example, for someone to be able to create, modify, or delete any\n# Instances, they must be granted project.collaborator, which means they can\n# create, modify, or delete _all_ resources in the Project.\n#\n# The complete set of predefined roles:\n#\n# - fleet.admin (superuser for the whole system)\n# - fleet.collaborator (can manage Silos)\n# - fleet.viewer (can read most non-siloed resources in the system)\n# - silo.admin (superuser for the silo)\n# - silo.collaborator (can create and own Organizations)\n# - silo.viewer (can read most resources within the Silo)\n# - organization.admin (complete control over an organization)\n# - organization.collaborator (can manage Projects)\n# - organization.viewer (can read most resources within the Organization)\n# - project.admin (complete control over a Project)\n# - project.collaborator (can manage all resources within the Project)\n# - project.viewer (can read most resources within the Project)\n#\n# Outside the Silo/Organization/Project hierarchy, we (currently) treat most\n# resources as nested under Fleet or else a synthetic resource (see below). We\n# do not yet support role assignments on anything other than Fleet, Silo,\n# Organization, or Project.\n#\n\n# "Fleet" is a global singleton representing the whole system. The name comes\n# from the idea described in RFD 24, but it's not quite right. This probably\n# should be more like "Region" or "AvailabilityZone". The precise boundaries\n# have not yet been figured out.\nresource Fleet {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\troles = [\n\t # Roles that can be attached by users\n\t "admin",\n\t "collaborator",\n\t "viewer",\n\n\t # Internal-only roles\n\t "external-authenticator"\n\t];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n}\n\n# For fleets specifically, roles can be conferred by roles on the user's Silo.\nhas_role(actor: AuthenticatedActor, role: String, _: Fleet) if\n\tsilo_role in actor.confers_fleet_role(role) and\n\thas_role(actor, silo_role, actor.silo.unwrap());\n\nresource Silo {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Permissions implied by roles on this resource's parent (Fleet). Fleet\n\t# privileges allow a user to see and potentially administer the Silo,\n\t# but they do not give anyone permission to look at anything inside the\n\t# Silo. To achieve this, we use permission rules here. (If we granted\n\t# Fleet administrators _roles_ on the Silo, then those would cascade\n\t# into the Silo as well.)\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "collaborator" on "parent_fleet";\n\n\t# external authenticator has to create silo users\n\t"list_children" if "external-authenticator" on "parent_fleet";\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", silo: Silo)\n\tif silo.fleet = fleet;\n\n# As a special case, all authenticated users can read their own Silo. That's\n# not quite the same as having the "viewer" role. For example, they cannot list\n# Organizations in the Silo.\n#\n# One reason this is necessary is because if an unprivileged user tries to\n# create an Organization using "POST /organizations", they should get back a 403\n# (which implies they're able to see /organizations, which is essentially seeing\n# the Silo itself) rather than a 404. This behavior isn't a hard constraint\n# (i.e., you could reasonably get a 404 for an API you're not allowed to call).\n# Nor is the implementation (i.e., we could special-case this endpoint somehow).\n# But granting this permission is the simplest way to keep this endpoint's\n# behavior consistent with the rest of the API.\n#\n# This rule is also used to determine if a user can list the identity providers\n# in the Silo (which they should be able to), since that's predicated on being\n# able to read the Silo.\n#\n# It's unclear what else would break if users couldn't see their own Silo.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif silo in actor.silo;\n\nresource Project {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\troles = [ "admin", "collaborator", "viewer" ];\n\n\t# Roles implied by other roles on this resource\n\t"viewer" if "collaborator";\n\t"collaborator" if "admin";\n\n\t# Permissions granted directly by roles on this resource\n\t"list_children" if "viewer";\n\t"read" if "viewer";\n\t"create_child" if "collaborator";\n\t"modify" if "admin";\n\n\t# Roles implied by roles on this resource's parent (Silo)\n\trelations = { parent_silo: Silo };\n\t"admin" if "collaborator" on "parent_silo";\n\t"viewer" if "viewer" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", project: Project)\n\tif project.silo = silo;\n\n#\n# GENERAL RESOURCES OUTSIDE THE SILO/PROJECT HIERARCHY\n#\n# Many resources use snippets of Polar generated by the `authz_resource!` Rust\n# macro. Some resources require custom Polar code. Those appear here.\n#\n\nresource Certificate {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Fleet-level and silo-level roles both grant privileges on certificates.\n\t"read" if "admin" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", certificate: Certificate)\n\tif certificate.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", certificate: Certificate)\n\tif certificate.silo.fleet = fleet;\n\nresource SiloUser {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\t# Fleet and Silo administrators can manage a Silo's users. This is one\n\t# of the only areas of Silo configuration that Fleet Administrators have\n\t# permissions on.\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "read" on "parent_fleet";\n\t"read" if "read" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", user: SiloUser)\n\tif user.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", user: SiloUser)\n\tif user.silo.fleet = fleet;\n\n# authenticated actors have all permissions on themselves\nhas_permission(actor: AuthenticatedActor, _perm: String, silo_user: SiloUser)\n if actor.equals_silo_user(silo_user);\n\nhas_permission(actor: AuthenticatedActor, "read", silo_user: SiloUser)\n if silo_user.silo in actor.silo;\n\nresource SiloGroup {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "read",\n\t "create_child",\n\t];\n\n\trelations = { parent_silo: Silo };\n\t"list_children" if "read" on "parent_silo";\n\t"read" if "read" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n}\nhas_relation(silo: Silo, "parent_silo", group: SiloGroup)\n\tif group.silo = silo;\n\nresource SshKey {\n\tpermissions = [ "read", "modify" ];\n\trelations = { silo_user: SiloUser };\n\n\t"read" if "read" on "silo_user";\n\t"modify" if "modify" on "silo_user";\n}\nhas_relation(user: SiloUser, "silo_user", ssh_key: SshKey)\n\tif ssh_key.silo_user = user;\n\nresource IdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", identity_provider: IdentityProvider)\n\tif identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: IdentityProvider)\n\tif collection.silo.fleet = fleet;\n\nresource SamlIdentityProvider {\n\tpermissions = [\n\t "read",\n\t "modify",\n\t "create_child",\n\t "list_children",\n\t];\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Silo-level roles grant privileges on identity providers.\n\t"read" if "viewer" on "parent_silo";\n\t"list_children" if "viewer" on "parent_silo";\n\t"modify" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_silo";\n\n\t# Fleet-level roles also grant privileges on identity providers.\n\t"read" if "viewer" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", saml_identity_provider: SamlIdentityProvider)\n\tif saml_identity_provider.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SamlIdentityProvider)\n\tif collection.silo.fleet = fleet;\n\n#\n# SYNTHETIC RESOURCES OUTSIDE THE SILO HIERARCHY\n#\n# The resources here do not correspond to anything that appears explicitly in\n# the API or is stored in the database. These are used either at the top level\n# of the API path (e.g., "/v1/system/ip-pools") or as an implementation detail of the system\n# (in the case of console sessions and "Database"). The policies are\n# either statically-defined in this file or driven by role assignments on the\n# Fleet. None of these resources defines their own roles.\n#\n\n# Describes the policy for reading and modifying DNS configuration\n# (both internal and external)\nresource DnsConfig {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t# "external-authenticator" requires these permissions because that's the\n\t# context that Nexus uses when creating and deleting Silos. These\n\t# operations necessarily need to read and modify DNS configuration.\n\t"read" if "external-authenticator" on "parent_fleet";\n\t"modify" if "external-authenticator" on "parent_fleet";\n\t# "admin" on the parent fleet also gets these permissions, primarily for\n\t# the test suite.\n\t"read" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", dns_config: DnsConfig)\n\tif dns_config.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource BlueprintConfig {\n\tpermissions = [\n\t "list_children", # list blueprints\n\t "create_child", # create blueprint\n\t "read", # read the current target\n\t "modify", # change the current target\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "admin" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n\t"list_children" if "viewer" on "parent_fleet";\n\t"read" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", list: BlueprintConfig)\n\tif list.fleet = fleet;\n\n# Describes the policy for accessing blueprints\nresource TargetReleaseConfig {\n\tpermissions = [\n\t "read", # read the current target release\n\t "modify", # change the current target release\n\t];\n\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", resource: TargetReleaseConfig)\n\tif resource.fleet = fleet;\n\n# Describes the policy for reading and modifying low-level inventory\nresource Inventory {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_fleet: Fleet };\n\t"read" if "viewer" on "parent_fleet";\n\t"modify" if "admin" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", inventory: Inventory)\n\tif inventory.fleet = fleet;\n\n# Describes the policy for accessing "/v1/system/ip-pools" in the API\nresource IpPoolList {\n\tpermissions = [\n\t "list_children",\n\t "modify",\n\t "create_child",\n\t];\n\n\t# Fleet Administrators can create or modify the IP Pools list.\n\trelations = { parent_fleet: Fleet };\n\t"modify" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n\n\t# Fleet Viewers can list IP Pools\n\t"list_children" if "viewer" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", ip_pool_list: IpPoolList)\n\tif ip_pool_list.fleet = fleet;\n\n# Any authenticated user can create a child of a provided IP Pool.\n# This is necessary to use the pools when provisioning instances.\nhas_permission(actor: AuthenticatedActor, "create_child", ip_pool: IpPool)\n\tif silo in actor.silo and silo.fleet = ip_pool.fleet;\n\n# Describes the policy for creating and managing web console sessions.\nresource ConsoleSessionList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: ConsoleSessionList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing device authorization requests.\nresource DeviceAuthRequestList {\n\tpermissions = [ "create_child" ];\n\trelations = { parent_fleet: Fleet };\n\t"create_child" if "external-authenticator" on "parent_fleet";\n}\nhas_relation(fleet: Fleet, "parent_fleet", collection: DeviceAuthRequestList)\n\tif collection.fleet = fleet;\n\n# Describes the policy for creating and managing Silo certificates\nresource SiloCertificateList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Both Fleet and Silo administrators can see and modify the Silo's\n\t# certificates.\n\t"list_children" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloCertificateList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloCertificateList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo identity providers\nresource SiloIdentityProviderList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the identity providers in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's identity provider\n\t# configuration. This is one of the only areas of Silo configuration\n\t# that Fleet Administrators have permissions on. This is also one of\n\t# the only cases where we need to look two levels up the hierarchy to\n\t# see if somebody has the right permission. For most other things,\n\t# permissions cascade down the hierarchy so we only need to look at the\n\t# parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloIdentityProviderList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloIdentityProviderList)\n\tif collection.silo.fleet = fleet;\n\n# Describes the policy for creating and managing Silo users (mostly intended for\n# API-managed users)\nresource SiloUserList {\n\tpermissions = [ "list_children", "create_child" ];\n\n\trelations = { parent_silo: Silo, parent_fleet: Fleet };\n\n\t# Everyone who can read the Silo (which includes all the users in the\n\t# Silo) can see the users in it.\n\t"list_children" if "read" on "parent_silo";\n\n\t# Fleet and Silo administrators can manage the Silo's users. This is\n\t# one of the only areas of Silo configuration that Fleet Administrators\n\t# have permissions on. This is also one of the few cases (so far) where\n\t# we need to look two levels up the hierarchy to see if somebody has the\n\t# right permission. For most other things, permissions cascade down the\n\t# hierarchy so we only need to look at the parent.\n\t"create_child" if "admin" on "parent_silo";\n\t"list_children" if "admin" on "parent_fleet";\n\t"create_child" if "admin" on "parent_fleet";\n}\nhas_relation(silo: Silo, "parent_silo", collection: SiloUserList)\n\tif collection.silo = silo;\nhas_relation(fleet: Fleet, "parent_fleet", collection: SiloUserList)\n\tif collection.silo.fleet = fleet;\n\n# These rules grants the external authenticator role the permissions it needs to\n# read silo users and modify their sessions. This is necessary for login to\n# work.\nhas_permission(actor: AuthenticatedActor, "read", silo: Silo)\n\tif has_role(actor, "external-authenticator", silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", user: SiloUser)\n\tif has_role(actor, "external-authenticator", user.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "read", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", group: SiloGroup)\n\tif has_role(actor, "external-authenticator", group.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\nhas_permission(actor: AuthenticatedActor, "modify", session: ConsoleSession)\n\tif has_role(actor, "external-authenticator", session.fleet);\n\n# All authenticated users can read and delete device authn requests because\n# by necessity these operations happen before we've figured out what user (or\n# even Silo) the device auth is associated with. Any user can claim a device\n# auth request with the right user code (that's how it works) -- it's the user\n# code and associated logic that prevents unauthorized access here.\nhas_permission(_actor: AuthenticatedActor, "read", _device_auth: DeviceAuthRequest);\nhas_permission(_actor: AuthenticatedActor, "modify", _device_auth: DeviceAuthRequest);\n\nhas_permission(actor: AuthenticatedActor, "read", device_token: DeviceAccessToken)\n\tif has_role(actor, "external-authenticator", device_token.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", identity_provider: IdentityProvider)\n\tif has_role(actor, "external-authenticator", identity_provider.silo.fleet);\n\nhas_permission(actor: AuthenticatedActor, "read", saml_identity_provider: SamlIdentityProvider)\n\tif has_role(actor, "external-authenticator", saml_identity_provider.silo.fleet);\n\n# Describes the policy for who can access the internal database.\nresource Database {\n\tpermissions = [\n\t # "query" is required to perform any query against the database,\n\t # whether a read or write query. This is checked when an operation\n\t # checks out a database connection from the connection pool.\n\t #\n\t # Any authenticated user gets this permission. There's generally\n\t # some other authz check involved in the database query. For\n\t # example, if you're querying the database to "read" a "Project", we\n\t # should also be checking that. So why do we do this at all? It's\n\t # a belt-and-suspenders measure so that if we somehow introduced an\n\t # unauthenticated code path that hits the database, it cannot be\n\t # used to DoS the database because we won't allow the operation to\n\t # make the query. (As long as the code path _is_ authenticated, we\n\t # can use throttling mechanisms to prevent DoS.)\n\t "query",\n\n\t # "modify" is required to populate database data that's delivered\n\t # with the system. It should also be required for schema changes,\n\t # when we support those. This is separate from "query" so that we\n\t # cannot accidentally invoke these code paths from API calls and\n\t # other general functions.\n\t "modify"\n\t];\n}\n\n# All authenticated users have the "query" permission on the database.\nhas_permission(_actor: AuthenticatedActor, "query", _resource: Database);\n\n# The "db-init" user is the only one with the "modify" permission.\nhas_permission(USER_DB_INIT: AuthenticatedActor, "modify", _resource: Database);\nhas_permission(USER_DB_INIT: AuthenticatedActor, "create_child", _resource: IpPoolList);\n# It also has "admin" on the internal silo to populate it with built-in resources.\n# TODO-completeness: actually limit to just internal silo and not all silos\nhas_role(USER_DB_INIT: AuthenticatedActor, "admin", _silo: Silo);\n\n# Allow the internal API admin permissions on all silos.\nhas_role(USER_INTERNAL_API: AuthenticatedActor, "admin", _silo: Silo);\n\nresource WebhookSecret {\n\tpermissions = [ "read", "modify" ];\n\trelations = { parent_webhook_receiver: WebhookReceiver };\n\n\t"read" if "read" on "parent_webhook_receiver";\n\t"modify" if "modify" on "parent_webhook_receiver";\n}\n\nhas_relation(rx: WebhookReceiver, "parent_webhook_receiver", secret: WebhookSecret)\n\tif secret.webhook_receiver = rx;\n\nresource WebhookEventClassList {\n\tpermissions = [ "list_children" ];\n\trelations = { parent_fleet: Fleet };\n\n\t"list_children" if "viewer" on "parent_fleet";\n}\n\nhas_relation(fleet: Fleet, "parent_fleet", collection: WebhookEventClassList)\n\tif collection.fleet = fleet;\n\n\n\n resource Disk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Disk)\n if child.project = parent;\n \n\n resource Snapshot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Snapshot)\n if child.project = parent;\n \n\n resource ProjectImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: ProjectImage)\n if child.project = parent;\n \n\n resource AffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AffinityGroup)\n if child.project = parent;\n \n\n resource AntiAffinityGroup {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: AntiAffinityGroup)\n if child.project = parent;\n \n\n resource Instance {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Instance)\n if child.project = parent;\n \n\n resource IpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: IpPool)\n if child.fleet = fleet;\n \n\n resource InstanceNetworkInterface {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Instance\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InstanceNetworkInterface)\n if has_relation(project, "containing_project", child.instance);\n\n has_relation(parent: Instance, "parent", child: InstanceNetworkInterface)\n if child.instance = parent;\n \n\n resource Vpc {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: Vpc)\n if child.project = parent;\n \n\n resource VpcRouter {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcRouter)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcRouter)\n if child.vpc = parent;\n \n\n resource InternetGateway {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGateway)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: InternetGateway)\n if child.vpc = parent;\n \n\n resource InternetGatewayIpPool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpPool)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpPool)\n if child.internet_gateway = parent;\n \n\n resource InternetGatewayIpAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: InternetGateway\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: InternetGatewayIpAddress)\n if has_relation(project, "containing_project", child.internet_gateway);\n\n has_relation(parent: InternetGateway, "parent", child: InternetGatewayIpAddress)\n if child.internet_gateway = parent;\n \n\n resource RouterRoute {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: VpcRouter\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: RouterRoute)\n if has_relation(project, "containing_project", child.vpc_router);\n\n has_relation(parent: VpcRouter, "parent", child: RouterRoute)\n if child.vpc_router = parent;\n \n\n resource VpcSubnet {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = {\n containing_project: Project,\n parent: Vpc\n };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(project: Project, "containing_project", child: VpcSubnet)\n if has_relation(project, "containing_project", child.vpc);\n\n has_relation(parent: Vpc, "parent", child: VpcSubnet)\n if child.vpc = parent;\n \n\n resource FloatingIp {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_project: Project };\n "list_children" if "viewer" on "containing_project";\n "read" if "viewer" on "containing_project";\n "modify" if "collaborator" on "containing_project";\n "create_child" if "collaborator" on "containing_project";\n }\n\n has_relation(parent: Project, "containing_project", child: FloatingIp)\n if child.project = parent;\n \n\n resource Image {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: Image)\n if child.silo = parent;\n \n\n resource SiloImage {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n\n relations = { containing_silo: Silo };\n "list_children" if "viewer" on "containing_silo";\n "read" if "viewer" on "containing_silo";\n "modify" if "collaborator" on "containing_silo";\n "create_child" if "collaborator" on "containing_silo";\n }\n\n has_relation(parent: Silo, "containing_silo", child: SiloImage)\n if child.silo = parent;\n \n\n resource AddressLot {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: AddressLot)\n if child.fleet = fleet;\n \n\n resource Blueprint {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Blueprint)\n if child.fleet = fleet;\n \n\n resource LoopbackAddress {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: LoopbackAddress)\n if child.fleet = fleet;\n \n\n\n resource ConsoleSession {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: ConsoleSession)\n if child.fleet = fleet;\n \n\n resource DeviceAuthRequest {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAuthRequest)\n if child.fleet = fleet;\n \n\n resource DeviceAccessToken {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: DeviceAccessToken)\n if child.fleet = fleet;\n \n\n resource PhysicalDisk {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: PhysicalDisk)\n if child.fleet = fleet;\n \n\n resource Rack {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Rack)\n if child.fleet = fleet;\n \n\n resource RoleBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: RoleBuiltin)\n if child.fleet = fleet;\n \n\n\n\n\n\n resource SupportBundle {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: SupportBundle)\n if child.fleet = fleet;\n \n\n\n\n resource Sled {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Sled)\n if child.fleet = fleet;\n \n\n resource TufRepo {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufRepo)\n if child.fleet = fleet;\n \n\n resource TufArtifact {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: TufArtifact)\n if child.fleet = fleet;\n \n\n resource WebhookEvent {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookEvent)\n if child.fleet = fleet;\n \n\n resource WebhookReceiver {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: WebhookReceiver)\n if child.fleet = fleet;\n \n\n\n resource Zpool {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Zpool)\n if child.fleet = fleet;\n \n\n resource Service {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: Service)\n if child.fleet = fleet;\n \n\n resource UserBuiltin {\n permissions = [\n "list_children",\n "modify",\n "read",\n "create_child",\n ];\n \n relations = { parent_fleet: Fleet };\n "list_children" if "viewer" on "parent_fleet";\n "read" if "viewer" on "parent_fleet";\n "modify" if "admin" on "parent_fleet";\n "create_child" if "admin" on "parent_fleet";\n }\n has_relation(fleet: Fleet, "parent_fleet", child: UserBuiltin)\n if child.fleet = fleet;\n
1052025-04-30T06:12:57.415ZINFOomicron-dev (ServerContext): Setting up resolver using DNS address: [::1]:48988
1062025-04-30T06:12:57.415ZINFOomicron-dev (DnsResolver): new DNS resolver addresses = [[::1]:48988]
1072025-04-30T06:12:57.419ZINFOomicron-dev (ServerContext): Setting up qorb database pool from a single host url = PostgresConfigWithUrl {\n url_raw: "postgresql://root@[::1]:60242/omicron?sslmode=disable",\n config: Config {\n user: Some(\n "root",\n ),\n password: None,\n dbname: Some(\n "omicron",\n ),\n options: None,\n application_name: None,\n ssl_mode: Disable,\n host: [\n Tcp(\n "::1",\n ),\n ],\n hostaddr: [],\n port: [\n 60242,\n ],\n connect_timeout: None,\n tcp_user_timeout: None,\n keepalives: true,\n keepalives_idle: 7200s,\n keepalives_interval: None,\n keepalives_retries: None,\n target_session_attrs: Any,\n channel_binding: Prefer,\n load_balance_hosts: Disable,\n },\n}
1082025-04-30T06:12:57.434ZDEBGomicron-dev (ServerContext): registered USDT probes
1092025-04-30T06:12:57.508ZINFOomicron-dev (nexus): Database schema version is up to date desired_version = 140.0.0 found_version = 140.0.0
1102025-04-30T06:12:57.518ZINFOomicron-dev (SEC): SEC running sec_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
1112025-04-30T06:12:58.120ZDEBGomicron-dev (clickhouse-client): registered USDT probes id = 7630253b-4062-4454-adc5-ca09da121554
1122025-04-30T06:12:58.130ZTRAComicron-dev (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1132025-04-30T06:12:58.130ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
1142025-04-30T06:12:58.134ZDEBGomicron-dev (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1152025-04-30T06:12:58.136ZDEBGomicron-dev (DataLoader): attempting to create built-in users
1162025-04-30T06:12:58.154ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1172025-04-30T06:12:58.154ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
1182025-04-30T06:12:58.154ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1192025-04-30T06:12:58.187ZINFOomicron-dev (DataLoader): created 5 built-in users
1202025-04-30T06:12:58.189ZTRAComicron-dev (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1212025-04-30T06:12:58.189ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
1222025-04-30T06:12:58.192ZDEBGomicron-dev (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1232025-04-30T06:12:58.193ZDEBGomicron-dev (DataLoader): attempting to create built-in roles
1242025-04-30T06:12:58.193ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1252025-04-30T06:12:58.193ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
1262025-04-30T06:12:58.193ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1272025-04-30T06:12:58.217ZINFOomicron-dev (DataLoader): created 10 built-in roles
1282025-04-30T06:12:58.217ZTRAComicron-dev (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1292025-04-30T06:12:58.217ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
1302025-04-30T06:12:58.221ZDEBGomicron-dev (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1312025-04-30T06:12:58.221ZDEBGomicron-dev (DataLoader): attempting to create built-in role assignments
1322025-04-30T06:12:58.221ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
1332025-04-30T06:12:58.221ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
1342025-04-30T06:12:58.222ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
1352025-04-30T06:12:58.254ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: f0 f5 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55977 req_id = aaada8b0-f378-4299-b31e-8a9ee4682462
1362025-04-30T06:12:58.255ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61685,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55977 req_id = aaada8b0-f378-4299-b31e-8a9ee4682462
1372025-04-30T06:12:58.282ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 61685, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55977 req_id = aaada8b0-f378-4299-b31e-8a9ee4682462
1382025-04-30T06:12:58.306ZTRAComicron-dev (dns): buffer buffer = Length: 74 (0x4a) bytes\n0000: 84 35 01 00 00 01 00 00 00 00 00 01 0b 5f 72 65 .5..........._re\n0010: 70 6f 2d 64 65 70 6f 74 04 5f 74 63 70 0d 63 6f po-depot._tcp.co\n0020: 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 ntrol-plane.oxid\n0030: 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 e.internal..!...\n0040: 00 29 04 d0 00 00 00 00 00 00 .)........ kind = dns peer_addr = [::1]:62132 req_id = 116cc437-9f9d-4b84-9414-1a12ade411ea
1392025-04-30T06:12:58.306ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33845,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_repo-depot._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_repo-depot._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 11,\n 95,\n 114,\n 101,\n 112,\n 111,\n 45,\n 100,\n 101,\n 112,\n 111,\n 116,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62132 req_id = 116cc437-9f9d-4b84-9414-1a12ade411ea
1402025-04-30T06:12:58.306ZINFOomicron-dev (MgsUpdateDriver): starting MgsUpdateDriver
1412025-04-30T06:12:58.306ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33845, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_repo-depot._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_repo-depot._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 11, 95, 114, 101, 112, 111, 45, 100, 101, 112, 111, 116, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_repo-depot._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62132 req_id = 116cc437-9f9d-4b84-9414-1a12ade411ea
1422025-04-30T06:12:58.309ZTRAComicron-dev (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: e8 1f 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:63042 req_id = 56383757-e8c9-4c33-8011-9d75d66ec96c
1432025-04-30T06:12:58.309ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59423,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63042 req_id = 56383757-e8c9-4c33-8011-9d75d66ec96c
1442025-04-30T06:12:58.310ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59423, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_crucible-pantry._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 16, 95, 99, 114, 117, 99, 105, 98, 108, 101, 45, 112, 97, 110, 116, 114, 121, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_crucible-pantry._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63042 req_id = 56383757-e8c9-4c33-8011-9d75d66ec96c
1452025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /bgtasks versions = all
1462025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /bgtasks/activate versions = all
1472025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /bgtasks/view/{bgtask_name} versions = all
1482025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /clickhouse/policy versions = all
1492025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /clickhouse/policy versions = all
1502025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stop-request versions = all
1512025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/downstairs/{downstairs_id}/stopped versions = all
1522025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair/{repair_id}/progress versions = all
1532025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-finish versions = all
1542025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /crucible/0/upstairs/{upstairs_id}/repair-start versions = all
1552025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /demo-saga versions = all
1562025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /demo-saga/{demo_saga_id}/complete versions = all
1572025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /deployment/blueprints/all versions = all
1582025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = DELETE path = /deployment/blueprints/all/{blueprint_id} versions = all
1592025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /deployment/blueprints/all/{blueprint_id} versions = all
1602025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /deployment/blueprints/import versions = all
1612025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /deployment/blueprints/regenerate versions = all
1622025-04-30T06:12:58.389ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /deployment/blueprints/target versions = all
1632025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /deployment/blueprints/target versions = all
1642025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = PUT path = /deployment/blueprints/target/enabled versions = all
1652025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /disk/{disk_id}/remove-read-only-parent versions = all
1662025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = PUT path = /disks/{disk_id} versions = all
1672025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /experimental/v1/system/support-bundles versions = all
1682025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /experimental/v1/system/support-bundles versions = all
1692025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
1702025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
1712025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
1722025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
1732025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
1742025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
1752025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
1762025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /instances/{instance_id}/migrate versions = all
1772025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /metrics/collectors versions = all
1782025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /metrics/collectors/{collector_id}/producers versions = all
1792025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /metrics/producers versions = all
1802025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /mgs-updates versions = all
1812025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /nat/ipv4/changeset/{from_gen} versions = all
1822025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /oximeter/read-policy versions = all
1832025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /oximeter/read-policy versions = all
1842025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /physical-disk/expunge versions = all
1852025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /probes/{sled} versions = all
1862025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = PUT path = /racks/{rack_id}/initialization-complete versions = all
1872025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /sagas versions = all
1882025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /sagas/{saga_id} versions = all
1892025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /sled-agents/{sled_id} versions = all
1902025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /sled-agents/{sled_id} versions = all
1912025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /sled-agents/{sled_id}/firewall-rules-update versions = all
1922025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /sleds/add versions = all
1932025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /sleds/expunge versions = all
1942025-04-30T06:12:58.390ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /sleds/uninitialized versions = all
1952025-04-30T06:12:58.391ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = PUT path = /switch/{switch_id} versions = all
1962025-04-30T06:12:58.391ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = GET path = /v1/ping versions = all
1972025-04-30T06:12:58.391ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = PUT path = /vmms/{propolis_id} versions = all
1982025-04-30T06:12:58.391ZDEBGomicron-dev (dropshot_internal): registered endpoint local_addr = [::1]:12221 method = POST path = /volume/{volume_id}/remove-read-only-parent versions = all
1992025-04-30T06:12:58.391ZINFOomicron-dev (dropshot_internal): listening local_addr = [::1]:12221
2002025-04-30T06:12:58.392ZDEBGomicron-dev (dropshot_internal): successfully registered DTrace USDT probes local_addr = [::1]:12221
2012025-04-30T06:12:58.514ZINFOomicron-dev (DataLoader): created 4 built-in role assignments
2022025-04-30T06:12:58.515ZTRAComicron-dev (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2032025-04-30T06:12:58.515ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2042025-04-30T06:12:58.518ZDEBGomicron-dev (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2052025-04-30T06:12:58.518ZDEBGomicron-dev (DataLoader): attempting to create built-in silos
2062025-04-30T06:12:58.518ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2072025-04-30T06:12:58.518ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2082025-04-30T06:12:58.519ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2092025-04-30T06:12:58.631ZINFOomicron-dev (DataLoader): created 2 built-in silos
2102025-04-30T06:12:58.631ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2112025-04-30T06:12:58.631ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2122025-04-30T06:12:58.632ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2132025-04-30T06:12:58.652ZTRAComicron-dev (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2142025-04-30T06:12:58.652ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2152025-04-30T06:12:58.654ZDEBGomicron-dev (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2162025-04-30T06:12:58.655ZDEBGomicron-dev (DataLoader): attempting to create built-in projects
2172025-04-30T06:12:58.657ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2182025-04-30T06:12:58.657ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2192025-04-30T06:12:58.658ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2202025-04-30T06:12:58.664ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2212025-04-30T06:12:58.665ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2222025-04-30T06:12:58.665ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2232025-04-30T06:12:58.665ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2242025-04-30T06:12:58.668ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2252025-04-30T06:12:58.673ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2262025-04-30T06:12:58.673ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2272025-04-30T06:12:58.673ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2282025-04-30T06:12:58.675ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2292025-04-30T06:12:58.766ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2302025-04-30T06:12:58.769ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2312025-04-30T06:12:58.770ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }
2322025-04-30T06:12:58.770ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2332025-04-30T06:12:58.770ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2342025-04-30T06:12:58.770ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2352025-04-30T06:12:58.770ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2362025-04-30T06:12:58.773ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2372025-04-30T06:12:58.773ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2382025-04-30T06:12:58.773ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2392025-04-30T06:12:58.774ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2402025-04-30T06:12:58.779ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2412025-04-30T06:12:58.785ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) } result = Ok(())
2422025-04-30T06:12:58.785ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2432025-04-30T06:12:58.785ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2442025-04-30T06:12:58.788ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2452025-04-30T06:12:58.817ZINFOomicron-dev (DataLoader): created built-in services project
2462025-04-30T06:12:58.818ZTRAComicron-dev (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2472025-04-30T06:12:58.818ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2482025-04-30T06:12:58.820ZDEBGomicron-dev (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2492025-04-30T06:12:58.820ZDEBGomicron-dev (DataLoader): attempting to create built-in VPCs
2502025-04-30T06:12:58.820ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2512025-04-30T06:12:58.820ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2522025-04-30T06:12:58.821ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2532025-04-30T06:12:58.824ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2542025-04-30T06:12:58.824ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2552025-04-30T06:12:58.825ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2562025-04-30T06:12:58.828ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2572025-04-30T06:12:58.828ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2582025-04-30T06:12:58.828ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2592025-04-30T06:12:58.828ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2602025-04-30T06:12:58.829ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2612025-04-30T06:12:58.831ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2622025-04-30T06:12:58.831ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2632025-04-30T06:12:58.831ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2642025-04-30T06:12:58.832ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2652025-04-30T06:12:58.835ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2662025-04-30T06:12:58.835ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2672025-04-30T06:12:58.835ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2682025-04-30T06:12:58.835ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2692025-04-30T06:12:58.839ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2702025-04-30T06:12:58.846ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2712025-04-30T06:12:58.846ZTRAComicron-dev (DataLoader): successful lookup of siloed resource "Project" using built-in user
2722025-04-30T06:12:58.847ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }
2732025-04-30T06:12:58.847ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2742025-04-30T06:12:58.847ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2752025-04-30T06:12:58.847ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2762025-04-30T06:12:58.847ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2772025-04-30T06:12:58.857ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2782025-04-30T06:12:58.857ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2792025-04-30T06:12:58.857ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2802025-04-30T06:12:58.859ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2812025-04-30T06:12:58.862ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
2822025-04-30T06:12:58.862ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2832025-04-30T06:12:58.862ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2842025-04-30T06:12:58.863ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2852025-04-30T06:12:58.868ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2862025-04-30T06:12:58.873ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) } result = Ok(())
2872025-04-30T06:12:58.873ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2882025-04-30T06:12:58.873ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2892025-04-30T06:12:58.874ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2902025-04-30T06:12:58.895ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
2912025-04-30T06:12:58.896ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
2922025-04-30T06:12:58.896ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2932025-04-30T06:12:58.896ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2942025-04-30T06:12:58.897ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2952025-04-30T06:12:58.904ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
2962025-04-30T06:12:58.905ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
2972025-04-30T06:12:58.905ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
2982025-04-30T06:12:58.905ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
2992025-04-30T06:12:58.907ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3002025-04-30T06:12:58.907ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3012025-04-30T06:12:58.908ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3022025-04-30T06:12:58.908ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3032025-04-30T06:12:58.912ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3042025-04-30T06:12:58.917ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3052025-04-30T06:12:58.917ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3062025-04-30T06:12:58.917ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3072025-04-30T06:12:58.918ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3082025-04-30T06:12:58.930ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3092025-04-30T06:12:58.930ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3102025-04-30T06:12:58.931ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3112025-04-30T06:12:58.941ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3122025-04-30T06:12:58.942ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3132025-04-30T06:12:58.942ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3142025-04-30T06:12:58.949ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }
3152025-04-30T06:12:58.949ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3162025-04-30T06:12:58.949ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3172025-04-30T06:12:58.949ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3182025-04-30T06:12:58.950ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3192025-04-30T06:12:58.953ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3202025-04-30T06:12:58.953ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3212025-04-30T06:12:58.953ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3222025-04-30T06:12:58.954ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3232025-04-30T06:12:58.956ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3242025-04-30T06:12:58.956ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3252025-04-30T06:12:58.956ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3262025-04-30T06:12:58.957ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3272025-04-30T06:12:58.959ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3282025-04-30T06:12:58.966ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") } result = Ok(())
3292025-04-30T06:12:58.966ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3302025-04-30T06:12:58.966ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3312025-04-30T06:12:58.967ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3322025-04-30T06:12:59.089ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3332025-04-30T06:12:59.089ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3342025-04-30T06:12:59.089ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3352025-04-30T06:12:59.089ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3362025-04-30T06:12:59.090ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3372025-04-30T06:12:59.096ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3382025-04-30T06:12:59.096ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3392025-04-30T06:12:59.096ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3402025-04-30T06:12:59.097ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3412025-04-30T06:12:59.101ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3422025-04-30T06:12:59.101ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3432025-04-30T06:12:59.101ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3442025-04-30T06:12:59.102ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3452025-04-30T06:12:59.105ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3462025-04-30T06:12:59.111ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3472025-04-30T06:12:59.111ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3482025-04-30T06:12:59.111ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3492025-04-30T06:12:59.111ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3502025-04-30T06:12:59.196ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
3512025-04-30T06:12:59.196ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3522025-04-30T06:12:59.196ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3532025-04-30T06:12:59.196ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3542025-04-30T06:12:59.197ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3552025-04-30T06:12:59.199ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3562025-04-30T06:12:59.200ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3572025-04-30T06:12:59.200ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3582025-04-30T06:12:59.200ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3592025-04-30T06:12:59.203ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3602025-04-30T06:12:59.203ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3612025-04-30T06:12:59.203ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3622025-04-30T06:12:59.204ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3632025-04-30T06:12:59.206ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3642025-04-30T06:12:59.214ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
3652025-04-30T06:12:59.214ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3662025-04-30T06:12:59.214ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3672025-04-30T06:12:59.215ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3682025-04-30T06:12:59.221ZDEBGomicron-dev (DataLoader): attempting to create built-in VPC firewall rules
3692025-04-30T06:12:59.222ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3702025-04-30T06:12:59.222ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3712025-04-30T06:12:59.223ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3722025-04-30T06:12:59.226ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3732025-04-30T06:12:59.226ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3742025-04-30T06:12:59.227ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3752025-04-30T06:12:59.229ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3762025-04-30T06:12:59.229ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3772025-04-30T06:12:59.230ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3782025-04-30T06:12:59.233ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3792025-04-30T06:12:59.233ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3802025-04-30T06:12:59.234ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3812025-04-30T06:12:59.234ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3822025-04-30T06:12:59.234ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3832025-04-30T06:12:59.237ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
3842025-04-30T06:12:59.237ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3852025-04-30T06:12:59.237ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3862025-04-30T06:12:59.238ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3872025-04-30T06:12:59.240ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
3882025-04-30T06:12:59.240ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3892025-04-30T06:12:59.240ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3902025-04-30T06:12:59.241ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3912025-04-30T06:12:59.243ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3922025-04-30T06:12:59.251ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
3932025-04-30T06:12:59.251ZTRAComicron-dev (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
3942025-04-30T06:12:59.251ZTRAComicron-dev (DataLoader): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
3952025-04-30T06:12:59.251ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
3962025-04-30T06:12:59.251ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
3972025-04-30T06:12:59.251ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
3982025-04-30T06:12:59.252ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
3992025-04-30T06:12:59.255ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4002025-04-30T06:12:59.255ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4012025-04-30T06:12:59.255ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4022025-04-30T06:12:59.256ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4032025-04-30T06:12:59.259ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4042025-04-30T06:12:59.260ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4052025-04-30T06:12:59.260ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4062025-04-30T06:12:59.261ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4072025-04-30T06:12:59.349ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4082025-04-30T06:12:59.374ZDEBGomicron-dev (DataLoader): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4092025-04-30T06:12:59.374ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4102025-04-30T06:12:59.374ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4112025-04-30T06:12:59.375ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4122025-04-30T06:12:59.390ZTRAComicron-dev (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4132025-04-30T06:12:59.391ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4142025-04-30T06:12:59.391ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4152025-04-30T06:12:59.391ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4162025-04-30T06:12:59.391ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4172025-04-30T06:12:59.417ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4182025-04-30T06:12:59.417ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4192025-04-30T06:12:59.417ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4202025-04-30T06:12:59.418ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4212025-04-30T06:12:59.423ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4222025-04-30T06:12:59.424ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4232025-04-30T06:12:59.424ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4242025-04-30T06:12:59.425ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4252025-04-30T06:12:59.431ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4262025-04-30T06:12:59.446ZDEBGomicron-dev (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4272025-04-30T06:12:59.447ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4282025-04-30T06:12:59.447ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4292025-04-30T06:12:59.450ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4302025-04-30T06:12:59.600ZINFOomicron-dev (DataLoader): created built-in VPC firewall rules
4312025-04-30T06:12:59.600ZDEBGomicron-dev (DataLoader): attempting to create built-in VPC Subnets
4322025-04-30T06:12:59.600ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4332025-04-30T06:12:59.600ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4342025-04-30T06:12:59.601ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4352025-04-30T06:12:59.607ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4362025-04-30T06:12:59.608ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4372025-04-30T06:12:59.609ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4382025-04-30T06:12:59.613ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4392025-04-30T06:12:59.613ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4402025-04-30T06:12:59.614ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4412025-04-30T06:12:59.618ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4422025-04-30T06:12:59.618ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4432025-04-30T06:12:59.618ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4442025-04-30T06:12:59.618ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4452025-04-30T06:12:59.623ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4462025-04-30T06:12:59.627ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4472025-04-30T06:12:59.628ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4482025-04-30T06:12:59.628ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4492025-04-30T06:12:59.630ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4502025-04-30T06:12:59.632ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4512025-04-30T06:12:59.632ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4522025-04-30T06:12:59.632ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4532025-04-30T06:12:59.633ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4542025-04-30T06:12:59.635ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4552025-04-30T06:12:59.641ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4562025-04-30T06:12:59.641ZTRAComicron-dev (DataLoader): successful lookup of siloed resource "Vpc" using built-in user
4572025-04-30T06:12:59.641ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4582025-04-30T06:12:59.641ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4592025-04-30T06:12:59.641ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4602025-04-30T06:12:59.651ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4612025-04-30T06:12:59.651ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4622025-04-30T06:12:59.651ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4632025-04-30T06:12:59.651ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4642025-04-30T06:12:59.652ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4652025-04-30T06:12:59.654ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4662025-04-30T06:12:59.654ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4672025-04-30T06:12:59.655ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4682025-04-30T06:12:59.655ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4692025-04-30T06:12:59.658ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4702025-04-30T06:12:59.658ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4712025-04-30T06:12:59.658ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4722025-04-30T06:12:59.660ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4732025-04-30T06:12:59.662ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4742025-04-30T06:12:59.667ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
4752025-04-30T06:12:59.675ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
4762025-04-30T06:12:59.675ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4772025-04-30T06:12:59.675ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4782025-04-30T06:12:59.675ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4792025-04-30T06:12:59.676ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4802025-04-30T06:12:59.678ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
4812025-04-30T06:12:59.678ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4822025-04-30T06:12:59.678ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4832025-04-30T06:12:59.679ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4842025-04-30T06:12:59.681ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
4852025-04-30T06:12:59.681ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4862025-04-30T06:12:59.681ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4872025-04-30T06:12:59.681ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4882025-04-30T06:12:59.683ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4892025-04-30T06:12:59.689ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
4902025-04-30T06:12:59.689ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4912025-04-30T06:12:59.689ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4922025-04-30T06:12:59.690ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4932025-04-30T06:12:59.700ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4942025-04-30T06:12:59.700ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
4952025-04-30T06:12:59.700ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
4962025-04-30T06:12:59.703ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
4972025-04-30T06:12:59.703ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
4982025-04-30T06:12:59.704ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
4992025-04-30T06:12:59.704ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5002025-04-30T06:12:59.708ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5012025-04-30T06:12:59.708ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5022025-04-30T06:12:59.708ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5032025-04-30T06:12:59.708ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5042025-04-30T06:12:59.708ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5052025-04-30T06:12:59.709ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5062025-04-30T06:12:59.709ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5072025-04-30T06:12:59.709ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5082025-04-30T06:12:59.710ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5092025-04-30T06:12:59.718ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5102025-04-30T06:12:59.723ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5112025-04-30T06:12:59.729ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5122025-04-30T06:12:59.729ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5132025-04-30T06:12:59.729ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5142025-04-30T06:12:59.729ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5152025-04-30T06:12:59.730ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5162025-04-30T06:12:59.733ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5172025-04-30T06:12:59.733ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5182025-04-30T06:12:59.733ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5192025-04-30T06:12:59.735ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5202025-04-30T06:12:59.741ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5212025-04-30T06:12:59.741ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5222025-04-30T06:12:59.741ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5232025-04-30T06:12:59.742ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5242025-04-30T06:12:59.755ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5252025-04-30T06:12:59.775ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5262025-04-30T06:12:59.775ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5272025-04-30T06:12:59.775ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5282025-04-30T06:12:59.777ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5292025-04-30T06:12:59.852ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5302025-04-30T06:12:59.853ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5312025-04-30T06:12:59.855ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5322025-04-30T06:12:59.877ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
5332025-04-30T06:12:59.877ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5342025-04-30T06:12:59.877ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5352025-04-30T06:12:59.877ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5362025-04-30T06:12:59.878ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5372025-04-30T06:12:59.885ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5382025-04-30T06:12:59.885ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5392025-04-30T06:12:59.885ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5402025-04-30T06:12:59.886ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5412025-04-30T06:12:59.891ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5422025-04-30T06:12:59.891ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5432025-04-30T06:12:59.891ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5442025-04-30T06:12:59.892ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5452025-04-30T06:12:59.895ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5462025-04-30T06:12:59.903ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
5472025-04-30T06:12:59.908ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) }
5482025-04-30T06:12:59.908ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
5492025-04-30T06:12:59.908ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5502025-04-30T06:12:59.908ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5512025-04-30T06:12:59.908ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5522025-04-30T06:12:59.960ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
5532025-04-30T06:12:59.960ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5542025-04-30T06:12:59.960ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5552025-04-30T06:12:59.961ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5562025-04-30T06:12:59.969ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5572025-04-30T06:12:59.969ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5582025-04-30T06:12:59.969ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5592025-04-30T06:12:59.970ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5602025-04-30T06:12:59.989ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5612025-04-30T06:13:00.014ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = VpcRouter { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-074c-4000-8000-000000000001, lookup_type: ById(001de000-074c-4000-8000-000000000001) } result = Ok(())
5622025-04-30T06:13:00.018ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5632025-04-30T06:13:00.032ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5642025-04-30T06:13:00.033ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5652025-04-30T06:13:00.049ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5662025-04-30T06:13:00.054ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5672025-04-30T06:13:00.055ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5682025-04-30T06:13:00.078ZINFOomicron-dev (DataLoader): created built-in services vpc subnets
5692025-04-30T06:13:00.078ZINFOomicron-dev (DataLoader): created built-in services vpc
5702025-04-30T06:13:00.078ZTRAComicron-dev (DataLoader): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5712025-04-30T06:13:00.078ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5722025-04-30T06:13:00.083ZDEBGomicron-dev (DataLoader): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5732025-04-30T06:13:00.083ZDEBGomicron-dev (DataLoader): attempting to create silo users
5742025-04-30T06:13:00.083ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5752025-04-30T06:13:00.083ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5762025-04-30T06:13:00.084ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5772025-04-30T06:13:00.114ZINFOomicron-dev (DataLoader): created 2 silo users
5782025-04-30T06:13:00.114ZDEBGomicron-dev (DataLoader): attempting to create silo user role assignments
5792025-04-30T06:13:00.114ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5802025-04-30T06:13:00.114ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5812025-04-30T06:13:00.115ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5822025-04-30T06:13:00.119ZINFOomicron-dev (DataLoader): created 2 silo user role assignments
5832025-04-30T06:13:00.119ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5842025-04-30T06:13:00.119ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5852025-04-30T06:13:00.122ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5862025-04-30T06:13:00.128ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5872025-04-30T06:13:00.128ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5882025-04-30T06:13:00.128ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5892025-04-30T06:13:00.142ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
5902025-04-30T06:13:00.142ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
5912025-04-30T06:13:00.142ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5922025-04-30T06:13:00.142ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5932025-04-30T06:13:00.143ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5942025-04-30T06:13:00.146ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5952025-04-30T06:13:00.153ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
5962025-04-30T06:13:00.154ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
5972025-04-30T06:13:00.154ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
5982025-04-30T06:13:00.154ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
5992025-04-30T06:13:00.202ZTRAComicron-dev (DataLoader): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList
6002025-04-30T06:13:00.202ZTRAComicron-dev (DataLoader): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6012025-04-30T06:13:00.202ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6022025-04-30T06:13:00.202ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
6032025-04-30T06:13:00.210ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6042025-04-30T06:13:00.213ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
6052025-04-30T06:13:00.216ZDEBGomicron-dev (DataLoader): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = IpPoolList result = Ok(())
6062025-04-30T06:13:00.216ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6072025-04-30T06:13:00.216ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
6082025-04-30T06:13:00.216ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6092025-04-30T06:13:00.301ZTRAComicron-dev (DataLoader): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database
6102025-04-30T06:13:00.301ZDEBGomicron-dev (DataLoader): roles roles = RoleSet { roles: {} }
6112025-04-30T06:13:00.302ZDEBGomicron-dev (DataLoader): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000001, .. }) resource = Database result = Ok(())
6122025-04-30T06:13:00.318ZINFOomicron-dev (nexus): populate complete
6132025-04-30T06:13:00.318ZINFOomicron-dev (nexus): activating background tasks
6142025-04-30T06:13:00.318ZDEBGomicron-dev (init_with_steps): Running step start_sled1
6152025-04-30T06:13:00.319ZINFOomicron-dev (omicron_sled_agent::sim::Server): setting up sled agent server sled_id = b6d65341-167c-41df-9b5c-41cded99c229
6162025-04-30T06:13:00.326ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 1 reason = Timeout
6172025-04-30T06:13:00.326ZDEBGomicron-dev (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
6182025-04-30T06:13:00.326ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
6192025-04-30T06:13:00.326ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6202025-04-30T06:13:00.327ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6212025-04-30T06:13:00.327ZDEBGomicron-dev (BackgroundTasks): activating background_task = bfd_manager iteration = 1 reason = Timeout
6222025-04-30T06:13:00.327ZDEBGomicron-dev (BackgroundTasks): activating background_task = metrics_producer_gc iteration = 1 reason = Timeout
6232025-04-30T06:13:00.327ZDEBGomicron-dev (BackgroundTasks): activating background_task = external_endpoints iteration = 1 reason = Timeout
6242025-04-30T06:13:00.327ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 1 reason = Timeout
6252025-04-30T06:13:00.328ZDEBGomicron-dev (BackgroundTasks): activating background_task = phantom_disks iteration = 1 reason = Timeout
6262025-04-30T06:13:00.328ZWARNomicron-dev (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config nor servers
6272025-04-30T06:13:00.328ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
6282025-04-30T06:13:00.328ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6292025-04-30T06:13:00.328ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
6302025-04-30T06:13:00.328ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
6312025-04-30T06:13:00.328ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 355.107\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6322025-04-30T06:13:00.328ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
6332025-04-30T06:13:00.328ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 1 reason = Timeout
6342025-04-30T06:13:00.328ZDEBGomicron-dev (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
6352025-04-30T06:13:00.328ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
6362025-04-30T06:13:00.328ZINFOomicron-dev (BackgroundTasks): Metric producer GC running background_task = metrics_producer_gc expiration = 2025-04-30 06:03:00.327676335 UTC
6372025-04-30T06:13:00.330ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database
6382025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): roles background_task = metrics_producer_gc roles = RoleSet { roles: {} }
6392025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 1 reason = Timeout
6402025-04-30T06:13:00.331ZWARNomicron-dev (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config nor servers
6412025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 47.865\u{b5}s iteration = 1 status = {"error":"no config nor servers"}
6422025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 1 reason = Timeout
6432025-04-30T06:13:00.331ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
6442025-04-30T06:13:00.331ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
6452025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
6462025-04-30T06:13:00.331ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6472025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 1 reason = Timeout
6482025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 1 reason = Timeout
6492025-04-30T06:13:00.331ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
6502025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
6512025-04-30T06:13:00.331ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet
6522025-04-30T06:13:00.331ZWARNomicron-dev (BackgroundTasks): Blueprint execution: skipped background_task = crdb_node_id_collector reason = no blueprint
6532025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 208.535\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6542025-04-30T06:13:00.331ZDEBGomicron-dev (BackgroundTasks): activating background_task = blueprint_executor iteration = 1 reason = Timeout
6552025-04-30T06:13:00.331ZWARNomicron-dev (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor reason = no blueprint
6562025-04-30T06:13:00.331ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = physical_disk_adoption resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6572025-04-30T06:13:00.332ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 28.998\u{b5}s iteration = 1 status = {"error":"no blueprint"}
6582025-04-30T06:13:00.332ZDEBGomicron-dev (BackgroundTasks): activating background_task = blueprint_loader iteration = 1 reason = Timeout
6592025-04-30T06:13:00.336ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
6602025-04-30T06:13:00.336ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
6612025-04-30T06:13:00.336ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 1 reason = Timeout
6622025-04-30T06:13:00.336ZDEBGomicron-dev (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
6632025-04-30T06:13:00.336ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
6642025-04-30T06:13:00.336ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = metrics_producer_gc resource = Database result = Ok(())
6652025-04-30T06:13:00.336ZDEBGomicron-dev (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 1 reason = Timeout
6662025-04-30T06:13:00.336ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
6672025-04-30T06:13:00.336ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
6682025-04-30T06:13:00.336ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 1 reason = Timeout
6692025-04-30T06:13:00.337ZDEBGomicron-dev (BackgroundTasks): activating background_task = support_bundle_collector iteration = 1 reason = Timeout
6702025-04-30T06:13:00.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
6712025-04-30T06:13:00.337ZDEBGomicron-dev (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 1 reason = Timeout
6722025-04-30T06:13:00.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database
6732025-04-30T06:13:00.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
6742025-04-30T06:13:00.337ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6752025-04-30T06:13:00.337ZDEBGomicron-dev (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
6762025-04-30T06:13:00.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
6772025-04-30T06:13:00.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
6782025-04-30T06:13:00.337ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
6792025-04-30T06:13:00.337ZDEBGomicron-dev (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
6802025-04-30T06:13:00.337ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6812025-04-30T06:13:00.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
6822025-04-30T06:13:00.337ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
6832025-04-30T06:13:00.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
6842025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6852025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
6862025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
6872025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
6882025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
6892025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): activating background_task = webhook_deliverator iteration = 1 reason = Timeout
6902025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 1 reason = Timeout
6912025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
6922025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
6932025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
6942025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): activating background_task = instance_reincarnation iteration = 1 reason = Timeout
6952025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
6962025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
6972025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
6982025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): activating background_task = instance_updater iteration = 1 reason = Timeout
6992025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
7002025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
7012025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database
7022025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
7032025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): roles background_task = webhook_deliverator roles = RoleSet { roles: {} }
7042025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
7052025-04-30T06:13:00.338ZTRAComicron-dev (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: 77 26 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 w&..........._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:57682 req_id = 7f503d72-9b7c-4e50-b317-155446cec6cc
7062025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
7072025-04-30T06:13:00.338ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 30502,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57682 req_id = 7f503d72-9b7c-4e50-b317-155446cec6cc
7082025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): activating background_task = service_firewall_rule_propagation iteration = 1 reason = Timeout
7092025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database
7102025-04-30T06:13:00.338ZDEBGomicron-dev (service-firewall-rule-progator): starting background task for service firewall rule propagation background_task = service_firewall_rule_propagation
7112025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
7122025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
7132025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_deliverator resource = Database result = Ok(())
7142025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
7152025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 1 reason = Timeout
7162025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
7172025-04-30T06:13:00.338ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 30502, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_nameservice._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 12, 95, 110, 97, 109, 101, 115, 101, 114, 118, 105, 99, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_nameservice._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57682 req_id = 7f503d72-9b7c-4e50-b317-155446cec6cc
7182025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
7192025-04-30T06:13:00.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
7202025-04-30T06:13:00.338ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_replacement iteration = 1 reason = Timeout
7212025-04-30T06:13:00.339ZTRAComicron-dev (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: ec 3c 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 .<..........._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58133 req_id = a7b7c1f0-d158-4699-a997-77a7bc10e2de
7222025-04-30T06:13:00.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
7232025-04-30T06:13:00.339ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7242025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
7252025-04-30T06:13:00.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
7262025-04-30T06:13:00.339ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60476,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58133 req_id = a7b7c1f0-d158-4699-a997-77a7bc10e2de
7272025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
7282025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
7292025-04-30T06:13:00.339ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 60476, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_external-dns._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 13, 95, 101, 120, 116, 101, 114, 110, 97, 108, 45, 100, 110, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_external-dns._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58133 req_id = a7b7c1f0-d158-4699-a997-77a7bc10e2de
7302025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 1 reason = Timeout
7312025-04-30T06:13:00.339ZWARNomicron-dev (BackgroundTasks): Blueprint rendezvous: skipped background_task = blueprint_rendezvous reason = no blueprint
7322025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 1 reason = Timeout
7332025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 1 reason = Timeout
7342025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 35.08\u{b5}s iteration = 1 status = {"error":"no blueprint"}
7352025-04-30T06:13:00.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
7362025-04-30T06:13:00.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
7372025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
7382025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
7392025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 1 reason = Timeout
7402025-04-30T06:13:00.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database
7412025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
7422025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
7432025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
7442025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activating background_task = v2p_manager iteration = 1 reason = Timeout
7452025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_replacement_driver iteration = 1 reason = Timeout
7462025-04-30T06:13:00.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
7472025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
7482025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
7492025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
7502025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
7512025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activating background_task = vpc_route_manager iteration = 1 reason = Timeout
7522025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 1 reason = Timeout
7532025-04-30T06:13:00.339ZINFOomicron-dev (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
7542025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): activating background_task = webhook_dispatcher iteration = 1 reason = Timeout
7552025-04-30T06:13:00.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
7562025-04-30T06:13:00.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
7572025-04-30T06:13:00.339ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7582025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = instance_watcher iteration = 1 reason = Timeout
7592025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
7602025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
7612025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database
7622025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
7632025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
7642025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database
7652025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7662025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = webhook_dispatcher roles = RoleSet { roles: {} }
7672025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
7682025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
7692025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
7702025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
7712025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
7722025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
7732025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
7742025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = tuf_artifact_replication iteration = 1 reason = Timeout
7752025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
7762025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
7772025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = abandoned_vmm_reaper iteration = 1 reason = Timeout
7782025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7792025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
7802025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
7812025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database result = Ok(())
7822025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
7832025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = decommissioned_disk_cleaner iteration = 1 reason = Timeout
7842025-04-30T06:13:00.340ZINFOomicron-dev (BackgroundTasks): Decommissioned disk cleaner running background_task = decommissioned_disk_cleaner
7852025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet
7862025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = decommissioned_disk_cleaner resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
7872025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database
7882025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
7892025-04-30T06:13:00.340ZWARNomicron-dev (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_internal dns_group = internal error = no record found for Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
7902025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 8.529605ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_nameservice._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
7912025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
7922025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = lookup_region_port iteration = 1 reason = Timeout
7932025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database
7942025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = abandoned_vmm_reaper roles = RoleSet { roles: {} }
7952025-04-30T06:13:00.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database
7962025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = lookup_region_port roles = RoleSet { roles: {} }
7972025-04-30T06:13:00.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = saga_recovery iteration = 1 reason = Timeout
7982025-04-30T06:13:00.340ZDEBGomicron-dev (SagaRecoverer): listing candidate sagas for recovery
7992025-04-30T06:13:00.340ZTRAComicron-dev (SagaRecoverer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database
8002025-04-30T06:13:00.340ZDEBGomicron-dev (SagaRecoverer): roles roles = RoleSet { roles: {} }
8012025-04-30T06:13:00.341ZWARNomicron-dev (BackgroundTasks): failed to lookup DNS servers background_task = dns_servers_external dns_group = external error = no record found for Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8022025-04-30T06:13:00.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
8032025-04-30T06:13:00.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
8042025-04-30T06:13:00.341ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 7.757095ms iteration = 1 status = {"error":"failed to read list of DNS servers: no record found for Query { name: Name(\\"_external-dns._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8052025-04-30T06:13:00.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
8062025-04-30T06:13:00.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = abandoned_vmm_reaper resource = Database result = Ok(())
8072025-04-30T06:13:00.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = lookup_region_port resource = Database result = Ok(())
8082025-04-30T06:13:00.341ZDEBGomicron-dev (SagaRecoverer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000003a8a, .. }) resource = Database result = Ok(())
8092025-04-30T06:13:00.342ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8102025-04-30T06:13:00.346ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
8112025-04-30T06:13:00.346ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
8122025-04-30T06:13:00.346ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
8132025-04-30T06:13:00.346ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
8142025-04-30T06:13:00.350ZINFOomicron-dev (BackgroundTasks): Metric producer GC complete (no errors) background_task = metrics_producer_gc expiration = 2025-04-30 06:03:00.327676335 UTC pruned = {}
8152025-04-30T06:13:00.350ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = metrics_producer_gc elapsed = 20.933689ms iteration = 1 status = {"expiration":"2025-04-30T06:03:00.327676335Z","pruned":[]}
8162025-04-30T06:13:00.350ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8172025-04-30T06:13:00.350ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
8182025-04-30T06:13:00.351ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: a3 ce 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:50211 req_id = 9ac936fe-8bc3-4fc5-92ba-a9e12328a74e
8192025-04-30T06:13:00.351ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41934,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50211 req_id = 9ac936fe-8bc3-4fc5-92ba-a9e12328a74e
8202025-04-30T06:13:00.352ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 41934, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50211 req_id = 9ac936fe-8bc3-4fc5-92ba-a9e12328a74e
8212025-04-30T06:13:00.352ZERROomicron-dev (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8222025-04-30T06:13:00.352ZERROomicron-dev (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = nat_v4_garbage_collector error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8232025-04-30T06:13:00.352ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = nat_v4_garbage_collector elapsed = 21.72193ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8242025-04-30T06:13:00.354ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 75 2e 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 u............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:62382 req_id = 1429f2ac-f617-40f3-821d-e6f7e56b364a
8252025-04-30T06:13:00.354ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29998,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62382 req_id = 1429f2ac-f617-40f3-821d-e6f7e56b364a
8262025-04-30T06:13:00.354ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 29998, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_dendrite._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 9, 95, 100, 101, 110, 100, 114, 105, 116, 101, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_dendrite._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62382 req_id = 1429f2ac-f617-40f3-821d-e6f7e56b364a
8272025-04-30T06:13:00.355ZERROomicron-dev (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8282025-04-30T06:13:00.355ZERROomicron-dev (BackgroundTasks): failed to resolve addresses for Dendrite services background_task = bfd_manager error = no record found for Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
8292025-04-30T06:13:00.355ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = bfd_manager elapsed = 27.611054ms iteration = 1 status = {"error":"failed to resolve addresses for Dendrite services: no record found for Query { name: Name(\\"_dendrite._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
8302025-04-30T06:13:00.376ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 48.802116ms iteration = 1 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
8312025-04-30T06:13:00.397ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
8322025-04-30T06:13:00.397ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8332025-04-30T06:13:00.397ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
8342025-04-30T06:13:00.397ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
8352025-04-30T06:13:00.398ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
8362025-04-30T06:13:00.457ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = webhook_deliverator elapsed = 125.326064ms iteration = 1 status = {"by_rx":{},"error":null}
8372025-04-30T06:13:00.462ZINFOomicron-dev (SledAgent): created simulated sled agent server = b6d65341-167c-41df-9b5c-41cded99c229 sim_mode = Auto sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8382025-04-30T06:13:00.493ZINFOomicron-dev (omicron_sled_agent::sim::SimulatedUpstairs): registered sled b6d65341-167c-41df-9b5c-41cded99c229 storage
8392025-04-30T06:13:00.517ZINFOomicron-dev (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmphP02JT/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8402025-04-30T06:13:00.518ZDEBGomicron-dev (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmphP02JT/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8412025-04-30T06:13:00.518ZINFOomicron-dev (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmp3PY4YD/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8422025-04-30T06:13:00.518ZDEBGomicron-dev (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmp3PY4YD/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8432025-04-30T06:13:00.792ZDEBGomicron-dev (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8442025-04-30T06:13:00.794ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8452025-04-30T06:13:00.795ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8462025-04-30T06:13:00.797ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
8472025-04-30T06:13:00.797ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
8482025-04-30T06:13:00.797ZDEBGomicron-dev (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
8492025-04-30T06:13:00.797ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
8502025-04-30T06:13:00.801ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
8512025-04-30T06:13:00.801ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
8522025-04-30T06:13:00.801ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
8532025-04-30T06:13:00.802ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
8542025-04-30T06:13:00.807ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8552025-04-30T06:13:00.812ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
8562025-04-30T06:13:00.812ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
8572025-04-30T06:13:00.812ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
8582025-04-30T06:13:00.812ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
8592025-04-30T06:13:00.983ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
8602025-04-30T06:13:00.983ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
8612025-04-30T06:13:00.986ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8622025-04-30T06:13:00.990ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
8632025-04-30T06:13:00.990ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
8642025-04-30T06:13:00.990ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
8652025-04-30T06:13:00.991ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
8662025-04-30T06:13:01.022ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
8672025-04-30T06:13:01.046ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8682025-04-30T06:13:01.047ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
8692025-04-30T06:13:01.050ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
8702025-04-30T06:13:01.050ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
8712025-04-30T06:13:01.050ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
8722025-04-30T06:13:01.051ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
8732025-04-30T06:13:01.102ZDEBGomicron-dev (dropshot (Repo Depot)): registered endpoint local_addr = [::1]:43978 method = GET path = /artifact/sha256/{sha256} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8742025-04-30T06:13:01.102ZINFOomicron-dev (dropshot (Repo Depot)): listening local_addr = [::1]:43978 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8752025-04-30T06:13:01.103ZDEBGomicron-dev (dropshot (Repo Depot)): successfully registered DTrace USDT probes local_addr = [::1]:43978 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
8762025-04-30T06:13:01.130ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8772025-04-30T06:13:01.134ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
8782025-04-30T06:13:01.135ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
8792025-04-30T06:13:01.135ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
8802025-04-30T06:13:01.135ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
8812025-04-30T06:13:01.135ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
8822025-04-30T06:13:01.135ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
8832025-04-30T06:13:01.140ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
8842025-04-30T06:13:01.147ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
8852025-04-30T06:13:01.147ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
8862025-04-30T06:13:01.147ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
8872025-04-30T06:13:01.157ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
8882025-04-30T06:13:01.157ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
8892025-04-30T06:13:01.158ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
8902025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /artifacts sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8912025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /artifacts/{sha256} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8922025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /artifacts/{sha256}/copy-from-depot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8932025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /artifacts-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8942025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /artifacts-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8952025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /boot-disk/{boot_disk}/os/write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8962025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /boot-disk/{boot_disk}/os/write/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8972025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = DELETE path = /boot-disk/{boot_disk}/os/write/status/{update_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8982025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /bootstore/status sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
8992025-04-30T06:13:01.166ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /datasets sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9002025-04-30T06:13:01.167ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /disks/{disk_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9012025-04-30T06:13:01.167ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /disks/{disk_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9022025-04-30T06:13:01.167ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /eip-gateways sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9032025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /inventory sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9042025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9052025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /network-bootstore-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9062025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /omicron-config sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9072025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /omicron-physical-disks sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9082025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /sled-identifiers sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9092025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /sled-role sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9102025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /sleds sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9112025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/dladm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9122025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/ipadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9132025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/logs/download/{zone} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9142025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/logs/zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9152025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/nvmeadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9162025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/pargs-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9172025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/pfiles-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9182025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/pstack-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9192025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/zfs-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9202025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/zoneadm-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9212025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support/zpool-info sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9222025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support-bundles/{zpool_id}/{dataset_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9232025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = DELETE path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9242025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9252025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9262025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9272025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9282025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/download/{file} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9292025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9302025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = HEAD path = /support-bundles/{zpool_id}/{dataset_id}/{support_bundle_id}/index sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9312025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /switch-ports sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9322025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /timesync sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9332025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = DELETE path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9342025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9352025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /v2p sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9362025-04-30T06:13:01.174ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = DELETE path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9372025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /vmms/{propolis_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9382025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /vmms/{propolis_id}/disks/{disk_id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9392025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = DELETE path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9402025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /vmms/{propolis_id}/external-ip sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9412025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /vmms/{propolis_id}/poke sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9422025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /vmms/{propolis_id}/poke-single-step sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9432025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /vmms/{propolis_id}/sim-migration-source sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9442025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9452025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /vmms/{propolis_id}/state sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9462025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /vpc/{vpc_id}/firewall/rules sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9472025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9482025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /vpc-routes sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9492025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /zones sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9502025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = POST path = /zones/bundle-cleanup sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9512025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9522025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = PUT path = /zones/bundle-cleanup/context sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9532025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /zones/bundle-cleanup/utilization sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9542025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /zones/bundles sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9552025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /zones/bundles/{zone_name} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9562025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = DELETE path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9572025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /zones/bundles/{zone_name}/{bundle_id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9582025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:61881 method = GET path = /zpools sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
9592025-04-30T06:13:01.175ZINFOomicron-dev (dropshot): listening local_addr = [::1]:61881 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9602025-04-30T06:13:01.175ZDEBGomicron-dev (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:61881 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9612025-04-30T06:13:01.175ZDEBGomicron-dev (omicron_sled_agent::sim::Server): contacting server nexus sled_id = b6d65341-167c-41df-9b5c-41cded99c229
9622025-04-30T06:13:01.175ZDEBGomicron-dev (NexusClient): client request body = Some(Body) method = POST sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = http://[::1]:12221/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
9632025-04-30T06:13:01.175ZINFOomicron-dev (dropshot_internal): accepted connection local_addr = [::1]:12221 remote_addr = [::1]:52413
9642025-04-30T06:13:01.177ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = POST remote_addr = [::1]:52413 req_id = e60eec50-0006-4347-8dc9-6453eff0b946 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
9652025-04-30T06:13:01.188ZINFOomicron-dev (nexus): registered sled agent sled_uuid = b6d65341-167c-41df-9b5c-41cded99c229
9662025-04-30T06:13:01.204ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
9672025-04-30T06:13:01.232ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 900.645226ms iteration = 1 status = {"errors":[],"requests_created_ok":[]}
9682025-04-30T06:13:01.301ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
9692025-04-30T06:13:01.301ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
9702025-04-30T06:13:01.302ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
9712025-04-30T06:13:01.322ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9722025-04-30T06:13:01.325ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
9732025-04-30T06:13:01.325ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
9742025-04-30T06:13:01.325ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
9752025-04-30T06:13:01.326ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
9762025-04-30T06:13:01.326ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
9772025-04-30T06:13:01.326ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
9782025-04-30T06:13:01.351ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
9792025-04-30T06:13:01.351ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
9802025-04-30T06:13:01.352ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
9812025-04-30T06:13:01.352ZDEBGomicron-dev (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9822025-04-30T06:13:01.355ZINFOomicron-dev (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0
9832025-04-30T06:13:01.355ZDEBGomicron-dev (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
9842025-04-30T06:13:01.355ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database
9852025-04-30T06:13:01.355ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
9862025-04-30T06:13:01.355ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Fleet result = Ok(())
9872025-04-30T06:13:01.356ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
9882025-04-30T06:13:01.357ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database
9892025-04-30T06:13:01.357ZDEBGomicron-dev (BackgroundTasks): roles background_task = decommissioned_disk_cleaner roles = RoleSet { roles: {} }
9902025-04-30T06:13:01.358ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = decommissioned_disk_cleaner resource = Database result = Ok(())
9912025-04-30T06:13:01.378ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
9922025-04-30T06:13:01.378ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
9932025-04-30T06:13:01.379ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
9942025-04-30T06:13:01.385ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 1.049977311s iteration = 1 status = {"errors":[],"garbage_collect_requested":[]}
9952025-04-30T06:13:01.418ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = lookup_region_port elapsed = 1.081158306s iteration = 1 status = {"errors":[],"found_port_ok":[]}
9962025-04-30T06:13:01.420ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
9972025-04-30T06:13:01.430ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
9982025-04-30T06:13:01.430ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
9992025-04-30T06:13:01.430ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
10002025-04-30T06:13:01.431ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
10012025-04-30T06:13:01.448ZWARNomicron-dev (BackgroundTasks): failed to read DNS config background_task = dns_config_internal dns_group = internal error = Internal Error: expected exactly one latest version for DNS group Internal, found 0
10022025-04-30T06:13:01.448ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 1.122152204s iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group Internal, found 0"}
10032025-04-30T06:13:01.448ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
10042025-04-30T06:13:01.448ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10052025-04-30T06:13:01.448ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
10062025-04-30T06:13:01.448ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10072025-04-30T06:13:01.449ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10082025-04-30T06:13:01.465ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 1.13082264s iteration = 1 status = {"errors":[],"finish_invoked_ok":[]}
10092025-04-30T06:13:01.467ZTRAComicron-dev (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = instance failed total_found = 0
10102025-04-30T06:13:01.467ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database
10112025-04-30T06:13:01.467ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_reincarnation roles = RoleSet { roles: {} }
10122025-04-30T06:13:01.467ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_reincarnation resource = Database result = Ok(())
10132025-04-30T06:13:01.476ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = switch_port_config_manager elapsed = 1.142775345s iteration = 1 status = {}
10142025-04-30T06:13:01.492ZWARNomicron-dev (BackgroundTasks): failed to read target blueprint background_task = blueprint_loader error = Internal Error: no target blueprint set
10152025-04-30T06:13:01.492ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 1.162068325s iteration = 1 status = {"error":"failed to read target blueprint: Internal Error: no target blueprint set"}
10162025-04-30T06:13:01.499ZTRAComicron-dev (BackgroundTasks): reprocessing 0 outdated webhook globs... background_task = webhook_dispatcher
10172025-04-30T06:13:01.499ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database
10182025-04-30T06:13:01.499ZDEBGomicron-dev (BackgroundTasks): roles background_task = webhook_dispatcher roles = RoleSet { roles: {} }
10192025-04-30T06:13:01.500ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = webhook_dispatcher resource = Database result = Ok(())
10202025-04-30T06:13:01.506ZDEBGomicron-dev (BackgroundTasks): Found abandoned VMMs background_task = abandoned_vmm_reaper count = 0 total = 0
10212025-04-30T06:13:01.506ZINFOomicron-dev (BackgroundTasks): Abandoned VMMs reaped background_task = abandoned_vmm_reaper sled_reservations_deleted = 0 vmms_already_deleted = 0 vmms_deleted = 0 vmms_found = 0
10222025-04-30T06:13:01.506ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = abandoned_vmm_reaper elapsed = 1.16984555s iteration = 1 status = {"errors":[],"sled_reservations_deleted":0,"vmms_already_deleted":0,"vmms_deleted":0,"vmms_found":0}
10232025-04-30T06:13:01.507ZINFOomicron-dev (BackgroundTasks): Physical Disk Adoption: Rack not yet initialized background_task = physical_disk_adoption rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
10242025-04-30T06:13:01.508ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 1.17797084s iteration = 1 status = {"error":"rack not yet initialized: c19a698f-c6f9-4a17-ae30-20d711b8f7dc"}
10252025-04-30T06:13:01.588ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10262025-04-30T06:13:01.588ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10272025-04-30T06:13:01.589ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10282025-04-30T06:13:01.589ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10292025-04-30T06:13:01.593ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
10302025-04-30T06:13:01.593ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
10312025-04-30T06:13:01.593ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
10322025-04-30T06:13:01.593ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
10332025-04-30T06:13:01.593ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10342025-04-30T06:13:01.598ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
10352025-04-30T06:13:01.679ZINFOomicron-dev (SagaRecoverer): listed in-progress sagas count = 0
10362025-04-30T06:13:01.680ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = saga_recovery elapsed = 1.343750304s iteration = 1 status = {"last_pass":{"Success":{"nfailed":0,"nfound":0,"nrecovered":0,"nremoved":0,"nskipped":0}},"ntotal_failures":0,"ntotal_finished":0,"ntotal_recovered":0,"ntotal_sec_errors_bad_state":0,"ntotal_sec_errors_missing":0,"ntotal_started":0,"recent_failures":[],"recent_recoveries":[]}
10372025-04-30T06:13:01.724ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
10382025-04-30T06:13:01.724ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
10392025-04-30T06:13:01.725ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 37 0a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 7............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53491 req_id = bbf20cf6-8155-4120-aeca-5b71adbd5d0d
10402025-04-30T06:13:01.725ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14090,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53491 req_id = bbf20cf6-8155-4120-aeca-5b71adbd5d0d
10412025-04-30T06:13:01.725ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 14090, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:53491 req_id = bbf20cf6-8155-4120-aeca-5b71adbd5d0d
10422025-04-30T06:13:01.730ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
10432025-04-30T06:13:01.730ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.399512687s iteration = 1 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
10442025-04-30T06:13:01.763ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
10452025-04-30T06:13:01.763ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
10462025-04-30T06:13:01.764ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
10472025-04-30T06:13:01.880ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
10482025-04-30T06:13:01.880ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10492025-04-30T06:13:01.880ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
10502025-04-30T06:13:01.880ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
10512025-04-30T06:13:01.881ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
10522025-04-30T06:13:01.885ZINFOomicron-dev (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0
10532025-04-30T06:13:01.885ZDEBGomicron-dev (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
10542025-04-30T06:13:01.885ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database
10552025-04-30T06:13:01.885ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
10562025-04-30T06:13:01.885ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
10572025-04-30T06:13:01.911ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10582025-04-30T06:13:01.915ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
10592025-04-30T06:13:01.915ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
10602025-04-30T06:13:01.915ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
10612025-04-30T06:13:01.916ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
10622025-04-30T06:13:02.120ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_replacement elapsed = 1.786153974s iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10632025-04-30T06:13:02.125ZWARNomicron-dev (BackgroundTasks): inventory collection is None background_task = service_zone_nat_tracker
10642025-04-30T06:13:02.125ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 1.793042099s iteration = 1 status = {"error":"inventory collection is None"}
10652025-04-30T06:13:02.141ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
10662025-04-30T06:13:02.148ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 1.813635389s iteration = 1 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
10672025-04-30T06:13:02.149ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
10682025-04-30T06:13:02.149ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
10692025-04-30T06:13:02.149ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
10702025-04-30T06:13:02.150ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
10712025-04-30T06:13:02.190ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
10722025-04-30T06:13:02.190ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
10732025-04-30T06:13:02.190ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
10742025-04-30T06:13:02.190ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
10752025-04-30T06:13:02.191ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
10762025-04-30T06:13:02.197ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 2 reason = Signaled
10772025-04-30T06:13:02.197ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
10782025-04-30T06:13:02.197ZINFOomicron-dev (dropshot_internal): request completed latency_us = 1020394 local_addr = [::1]:12221 method = POST remote_addr = [::1]:52413 req_id = e60eec50-0006-4347-8dc9-6453eff0b946 response_code = 204 uri = /sled-agents/b6d65341-167c-41df-9b5c-41cded99c229
10792025-04-30T06:13:02.197ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
10802025-04-30T06:13:02.197ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
10812025-04-30T06:13:02.197ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
10822025-04-30T06:13:02.198ZDEBGomicron-dev (NexusClient): client response result = Ok(Response { url: "http://[::1]:12221/sled-agents/b6d65341-167c-41df-9b5c-41cded99c229", status: 204, headers: {"x-request-id": "e60eec50-0006-4347-8dc9-6453eff0b946", "date": "Wed, 30 Apr 2025 06:13:02 GMT"} }) sled_id = b6d65341-167c-41df-9b5c-41cded99c229
10832025-04-30T06:13:02.198ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
10842025-04-30T06:13:02.198ZDEBGomicron-dev (init_with_steps): Running step start_crucible_pantry
10852025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = GET path = /crucible/pantry/0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10862025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = GET path = /crucible/pantry/0/job/{id}/is-finished sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10872025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = GET path = /crucible/pantry/0/job/{id}/ok sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10882025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = DELETE path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10892025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = GET path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10902025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = POST path = /crucible/pantry/0/volume/{id} sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10912025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = POST path = /crucible/pantry/0/volume/{id}/background sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10922025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = POST path = /crucible/pantry/0/volume/{id}/bulk-write sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10932025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = POST path = /crucible/pantry/0/volume/{id}/import-from-url sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10942025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = POST path = /crucible/pantry/0/volume/{id}/scrub sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10952025-04-30T06:13:02.201ZDEBGomicron-dev (dropshot): registered endpoint kind = pantry local_addr = [::1]:55020 method = POST path = /crucible/pantry/0/volume/{id}/snapshot sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
10962025-04-30T06:13:02.201ZDEBGomicron-dev (BackgroundTasks): Found zpools on decommissioned disks background_task = decommissioned_disk_cleaner count = 0
10972025-04-30T06:13:02.201ZINFOomicron-dev (BackgroundTasks): Cleaned decommissioned zpools background_task = decommissioned_disk_cleaner deleted = 0 error_count = 0 found = 0 not_ready_to_be_deleted = 0
10982025-04-30T06:13:02.202ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = decommissioned_disk_cleaner elapsed = 1.865445357s iteration = 1 status = {"deleted":0,"error":null,"error_count":0,"found":0,"not_ready_to_be_deleted":0}
10992025-04-30T06:13:02.202ZINFOomicron-dev (dropshot): listening kind = pantry local_addr = [::1]:55020 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11002025-04-30T06:13:02.204ZDEBGomicron-dev (dropshot): successfully registered DTrace USDT probes kind = pantry local_addr = [::1]:55020 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11012025-04-30T06:13:02.204ZINFOomicron-dev (omicron_sled_agent::sim::Server): Started Simulated Crucible Pantry address = [::1]:55020 kind = pantry sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11022025-04-30T06:13:02.204ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11032025-04-30T06:13:02.204ZDEBGomicron-dev (init_with_steps): Running step populate_internal_dns
11042025-04-30T06:13:02.204ZDEBGomicron-dev: Populating Internal DNS
11052025-04-30T06:13:02.206ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
11062025-04-30T06:13:02.206ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
11072025-04-30T06:13:02.207ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
11082025-04-30T06:13:02.207ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
11092025-04-30T06:13:02.215ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11102025-04-30T06:13:02.215ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11112025-04-30T06:13:02.215ZTRAComicron-dev (BackgroundTasks): webhook dispatching completed successfully background_task = webhook_dispatcher events_dispatched = 0 events_without_receivers = 0
11122025-04-30T06:13:02.215ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = webhook_dispatcher elapsed = 1.880338711s iteration = 1 status = {"dispatched":[],"errors":[],"glob_version":"140.0.0","globs_reprocessed":{},"no_receivers":[]}
11132025-04-30T06:13:02.215ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11142025-04-30T06:13:02.228ZINFOomicron-dev (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
11152025-04-30T06:13:02.228ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 1.893104917s iteration = 1 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
11162025-04-30T06:13:02.237ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
11172025-04-30T06:13:02.237ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11182025-04-30T06:13:02.237ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11192025-04-30T06:13:02.238ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11202025-04-30T06:13:02.337ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11212025-04-30T06:13:02.342ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
11222025-04-30T06:13:02.342ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
11232025-04-30T06:13:02.342ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
11242025-04-30T06:13:02.343ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
11252025-04-30T06:13:02.354ZINFOomicron-dev: DNS population: DnsConfigParams { generation: Generation( 1, ), time_created: 2025-04-30T06:13:02.353636026Z, zones: [ DnsConfigZone { zone_name: "control-plane.oxide.internal", records: { "e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c.host": [ Aaaa( ::1, ), ], "_cockroach._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 60242, target: "bb6e149b-0d6f-4f9e-8216-e4a59d897bce.host.control-plane.oxide.internal", }, ), ], "_nameservice._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 39475, target: "cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal", }, ), ], "_external-dns._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 54696, target: "90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal", }, ), ], "_mgd._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 54778, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "bb6e149b-0d6f-4f9e-8216-e4a59d897bce.host": [ Aaaa( ::1, ), ], "_repo-depot._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 43978, target: "b6d65341-167c-41df-9b5c-41cded99c229.sled.control-plane.oxide.internal", }, ), ], "510be8b8-7e77-4044-896c-ee4a50926337.host": [ Aaaa( ::1, ), ], "_mgs._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 40478, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_clickhouse-admin-single-server._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 8888, target: "510be8b8-7e77-4044-896c-ee4a50926337.host.control-plane.oxide.internal", }, ), ], "_nexus._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 12221, target: "e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c.host.control-plane.oxide.internal", }, ), ], "_clickhouse-native._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "510be8b8-7e77-4044-896c-ee4a50926337.host.control-plane.oxide.internal", }, ), ], "_oximeter-reader._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 9000, target: "510be8b8-7e77-4044-896c-ee4a50926337.host.control-plane.oxide.internal", }, ), ], "_dendrite._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 46156, target: "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal", }, ), ], "_clickhouse._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 58355, target: "510be8b8-7e77-4044-896c-ee4a50926337.host.control-plane.oxide.internal", }, ), ], "_crucible-pantry._tcp": [ Srv( Srv { prio: 0, weight: 0, port: 55020, target: "8353cc71-8d6e-45ed-b15b-a3105d4a652c.host.control-plane.oxide.internal", }, ), ], "dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host": [ Aaaa( ::1, ), ], "b6d65341-167c-41df-9b5c-41cded99c229.sled": [ Aaaa( ::1, ), ], "8353cc71-8d6e-45ed-b15b-a3105d4a652c.host": [ Aaaa( ::1, ), ], "90372d2a-e95e-4302-b338-2843dd07d33a.host": [ Aaaa( ::1, ), ], "cd92814a-77d2-4d97-8e60-d67fa56f9567.host": [ Aaaa( ::1, ), ], }, }, ], }
11262025-04-30T06:13:02.385ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
11272025-04-30T06:13:02.385ZDEBGomicron-dev: client request body = Some(Body) method = PUT uri = http://[::1]:39475/config
11282025-04-30T06:13:02.385ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11292025-04-30T06:13:02.385ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11302025-04-30T06:13:02.385ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11312025-04-30T06:13:02.386ZINFOomicron-dev (http): accepted connection kind = dns local_addr = [::1]:39475 remote_addr = [::1]:41735
11322025-04-30T06:13:02.386ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11332025-04-30T06:13:02.447ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11342025-04-30T06:13:02.447ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11352025-04-30T06:13:02.447ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11362025-04-30T06:13:02.467ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11372025-04-30T06:13:02.487ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11382025-04-30T06:13:02.492ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
11392025-04-30T06:13:02.492ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11402025-04-30T06:13:02.492ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11412025-04-30T06:13:02.492ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11422025-04-30T06:13:02.517ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
11432025-04-30T06:13:02.517ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
11442025-04-30T06:13:02.518ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: e8 b7 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54364 req_id = 3ae55c31-a53b-4f24-be3a-0a35f32c2348
11452025-04-30T06:13:02.518ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59575,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54364 req_id = 3ae55c31-a53b-4f24-be3a-0a35f32c2348
11462025-04-30T06:13:02.518ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 59575, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_mgs._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 4, 95, 109, 103, 115, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } SERVFAIL: server is not authoritative for name: "_mgs._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54364 req_id = 3ae55c31-a53b-4f24-be3a-0a35f32c2348
11472025-04-30T06:13:02.518ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN }
11482025-04-30T06:13:02.519ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 321.437113ms iteration = 2 status = {"error":"failed to collect inventory: looking up MGS addresses: no record found for Query { name: Name(\\"_mgs._tcp.control-plane.oxide.internal.\\"), query_type: SRV, query_class: IN }"}
11492025-04-30T06:13:02.537ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 2.202834393s iteration = 1 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
11502025-04-30T06:13:02.542ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11512025-04-30T06:13:02.545ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 2.210910785s iteration = 1 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
11522025-04-30T06:13:02.546ZWARNomicron-dev (BackgroundTasks): failed to read DNS config background_task = dns_config_external dns_group = external error = Internal Error: expected exactly one latest version for DNS group External, found 0
11532025-04-30T06:13:02.546ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 2.216013691s iteration = 1 status = {"error":"failed to read DNS config: Internal Error: expected exactly one latest version for DNS group External, found 0"}
11542025-04-30T06:13:02.549ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11552025-04-30T06:13:02.549ZTRAComicron-dev (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
11562025-04-30T06:13:02.549ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
11572025-04-30T06:13:02.549ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
11582025-04-30T06:13:02.549ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11592025-04-30T06:13:02.549ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11602025-04-30T06:13:02.549ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11612025-04-30T06:13:02.550ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet
11622025-04-30T06:13:02.550ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = tuf_artifact_replication resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11632025-04-30T06:13:02.550ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11642025-04-30T06:13:02.550ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11652025-04-30T06:13:02.551ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11662025-04-30T06:13:02.553ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
11672025-04-30T06:13:02.553ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11682025-04-30T06:13:02.553ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11692025-04-30T06:13:02.553ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11702025-04-30T06:13:02.553ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11712025-04-30T06:13:02.556ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11722025-04-30T06:13:02.556ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11732025-04-30T06:13:02.556ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11742025-04-30T06:13:02.556ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11752025-04-30T06:13:02.557ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Fleet result = Ok(())
11762025-04-30T06:13:02.557ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database
11772025-04-30T06:13:02.557ZDEBGomicron-dev (BackgroundTasks): roles background_task = tuf_artifact_replication roles = RoleSet { roles: {} }
11782025-04-30T06:13:02.558ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = tuf_artifact_replication resource = Database result = Ok(())
11792025-04-30T06:13:02.561ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
11802025-04-30T06:13:02.567ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
11812025-04-30T06:13:02.567ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
11822025-04-30T06:13:02.567ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
11832025-04-30T06:13:02.567ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
11842025-04-30T06:13:02.568ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
11852025-04-30T06:13:02.569ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication body = Some(Body) method = PUT uri = http://[::1]:61881/artifacts-config
11862025-04-30T06:13:02.569ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:39820 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
11872025-04-30T06:13:02.569ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
11882025-04-30T06:13:02.569ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
11892025-04-30T06:13:02.569ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
11902025-04-30T06:13:02.570ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
11912025-04-30T06:13:02.570ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = PUT remote_addr = [::1]:39820 req_id = e1cf3761-3a3a-4103-9776-e5e3e3d419ed sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts-config
11922025-04-30T06:13:02.572ZINFOomicron-dev (BackgroundTasks): No bundles to collect background_task = support_bundle_collector
11932025-04-30T06:13:02.573ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 2.24190532s iteration = 1 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
11942025-04-30T06:13:02.573ZTRAComicron-dev (http): incoming request kind = dns local_addr = [::1]:39475 method = PUT remote_addr = [::1]:41735 req_id = bb9edc96-0259-4805-ad2f-53a2897c0981 uri = /config
11952025-04-30T06:13:02.574ZINFOomicron-dev (store): attempting generation update new_generation = 1 req_id = bb9edc96-0259-4805-ad2f-53a2897c0981
11962025-04-30T06:13:02.574ZINFOomicron-dev (store): pruning trees for generations newer than 0
11972025-04-30T06:13:02.574ZDEBGomicron-dev (store): creating tree tree_name = generation_1_zone_control-plane.oxide.internal
11982025-04-30T06:13:02.575ZDEBGomicron-dev (store): updating current config
11992025-04-30T06:13:02.575ZDEBGomicron-dev (store): flushing default tree
12002025-04-30T06:13:02.576ZINFOomicron-dev (store): pruning trees for generations older than 1 keep = 3
12012025-04-30T06:13:02.576ZINFOomicron-dev (store): updated generation new_generation = 1 req_id = bb9edc96-0259-4805-ad2f-53a2897c0981
12022025-04-30T06:13:02.576ZINFOomicron-dev (http): request completed kind = dns latency_us = 2437 local_addr = [::1]:39475 method = PUT remote_addr = [::1]:41735 req_id = bb9edc96-0259-4805-ad2f-53a2897c0981 response_code = 204 uri = /config
12032025-04-30T06:13:02.576ZINFOomicron-dev (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmphP02JT/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12042025-04-30T06:13:02.576ZDEBGomicron-dev (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmphP02JT/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12052025-04-30T06:13:02.576ZINFOomicron-dev (ArtifactStore): No ledger in /var/tmp/omicron_tmp/.tmp3PY4YD/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12062025-04-30T06:13:02.576ZDEBGomicron-dev (ArtifactStore): Failed to read ledger: Not found in storage path = /var/tmp/omicron_tmp/.tmp3PY4YD/artifact-config.json server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12072025-04-30T06:13:02.577ZINFOomicron-dev (ArtifactStore): Writing ledger to /var/tmp/omicron_tmp/.tmphP02JT/.artifact-config.json.tmp server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12082025-04-30T06:13:02.577ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12092025-04-30T06:13:02.577ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12102025-04-30T06:13:02.578ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12112025-04-30T06:13:02.578ZINFOomicron-dev (ArtifactStore): Writing ledger to /var/tmp/omicron_tmp/.tmp3PY4YD/.artifact-config.json.tmp server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12122025-04-30T06:13:02.579ZINFOomicron-dev (ArtifactStore): Starting delete reconciler generation = 1 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12132025-04-30T06:13:02.579ZINFOomicron-dev (dropshot): request completed latency_us = 8283 local_addr = [::1]:61881 method = PUT remote_addr = [::1]:39820 req_id = e1cf3761-3a3a-4103-9776-e5e3e3d419ed response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts-config
12142025-04-30T06:13:02.579ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[::1]:61881/artifacts-config", status: 204, headers: {"x-request-id": "e1cf3761-3a3a-4103-9776-e5e3e3d419ed", "date": "Wed, 30 Apr 2025 06:13:02 GMT"} })
12152025-04-30T06:13:02.580ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication body = None method = GET uri = http://[::1]:61881/artifacts
12162025-04-30T06:13:02.580ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12172025-04-30T06:13:02.580ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:39820 req_id = c0b4c099-2e8f-40c3-8a4b-6eeecad7212f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts
12182025-04-30T06:13:02.580ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12192025-04-30T06:13:02.581ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12202025-04-30T06:13:02.581ZINFOomicron-dev (dropshot): request completed latency_us = 678 local_addr = [::1]:61881 method = GET remote_addr = [::1]:39820 req_id = c0b4c099-2e8f-40c3-8a4b-6eeecad7212f response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /artifacts
12212025-04-30T06:13:02.581ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = tuf_artifact_replication result = Ok(Response { url: "http://[::1]:61881/artifacts", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0b4c099-2e8f-40c3-8a4b-6eeecad7212f", "content-length": "26", "date": "Wed, 30 Apr 2025 06:13:02 GMT"} })
12222025-04-30T06:13:02.582ZINFOomicron-dev (BackgroundTasks): Successfully got artifact list background_task = tuf_artifact_replication sled = http://[::1]:61881
12232025-04-30T06:13:02.582ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12242025-04-30T06:13:02.583ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12252025-04-30T06:13:02.583ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12262025-04-30T06:13:02.584ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12272025-04-30T06:13:02.585ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = tuf_artifact_replication elapsed = 2.249217608s iteration = 1 status = {"generation":1,"last_run_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"lifetime_counters":{"copy_err":0,"copy_ok":0,"list_err":0,"list_ok":1,"put_config_err":0,"put_config_ok":1,"put_err":0,"put_ok":0},"local_repos":0,"request_debug_ringbuf":[{"generation":1,"operation":"put_config","target_sled":"b6d65341-167c-41df-9b5c-41cded99c229","time":"2025-04-30T06:13:02.579944702Z"},{"operation":"list","target_sled":"b6d65341-167c-41df-9b5c-41cded99c229","time":"2025-04-30T06:13:02.582760522Z"}]}
12282025-04-30T06:13:02.586ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12292025-04-30T06:13:02.586ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
12302025-04-30T06:13:02.587ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12312025-04-30T06:13:02.587ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
12322025-04-30T06:13:02.587ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12332025-04-30T06:13:02.587ZDEBGomicron-dev: client response result = Ok(Response { url: "http://[::1]:39475/config", status: 204, headers: {"x-request-id": "bb9edc96-0259-4805-ad2f-53a2897c0981", "date": "Wed, 30 Apr 2025 06:13:02 GMT"} })
12342025-04-30T06:13:02.587ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12352025-04-30T06:13:02.593ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
12362025-04-30T06:13:02.593ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
12372025-04-30T06:13:02.593ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12382025-04-30T06:13:02.593ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12392025-04-30T06:13:02.594ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12402025-04-30T06:13:02.599ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12412025-04-30T06:13:02.599ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
12422025-04-30T06:13:02.599ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
12432025-04-30T06:13:02.599ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
12442025-04-30T06:13:02.600ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
12452025-04-30T06:13:02.603ZWARNomicron-dev (BackgroundTasks): failed to read Silo/DNS/TLS configuration background_task = external_endpoints error = Internal Error: expected at least one external DNS zone
12462025-04-30T06:13:02.603ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 2.276018147s iteration = 1 status = {"error":"failed to read Silo/DNS/TLS configuration: Internal Error: expected at least one external DNS zone"}
12472025-04-30T06:13:02.605ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
12482025-04-30T06:13:02.605ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12492025-04-30T06:13:02.605ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12502025-04-30T06:13:02.605ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12512025-04-30T06:13:02.607ZDEBGomicron-dev (init_with_steps): Running step configure_sled_agents
12522025-04-30T06:13:02.607ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12532025-04-30T06:13:02.607ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12542025-04-30T06:13:02.607ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12552025-04-30T06:13:02.608ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12562025-04-30T06:13:02.611ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
12572025-04-30T06:13:02.617ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
12582025-04-30T06:13:02.617ZTRAComicron-dev (BackgroundTasks): successful lookup of siloed resource "Vpc" using built-in user background_task = service_firewall_rule_propagation
12592025-04-30T06:13:02.617ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12602025-04-30T06:13:02.617ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12612025-04-30T06:13:02.618ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12622025-04-30T06:13:02.638ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12632025-04-30T06:13:02.638ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12642025-04-30T06:13:02.639ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12652025-04-30T06:13:02.662ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12662025-04-30T06:13:02.662ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12672025-04-30T06:13:02.665ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12682025-04-30T06:13:02.742ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12692025-04-30T06:13:02.742ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12702025-04-30T06:13:02.743ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12712025-04-30T06:13:02.758ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
12722025-04-30T06:13:02.759ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
12732025-04-30T06:13:02.759ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12742025-04-30T06:13:02.759ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12752025-04-30T06:13:02.759ZDEBGomicron-dev: client request body = Some(Body) method = PUT uri = http://[::1]:61881/omicron-config
12762025-04-30T06:13:02.760ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12772025-04-30T06:13:02.760ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:33008 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
12782025-04-30T06:13:02.762ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = PUT remote_addr = [::1]:33008 req_id = a8ac3d69-8b35-42b8-994e-003bb175c3f0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-config
12792025-04-30T06:13:02.763ZINFOomicron-dev (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0
12802025-04-30T06:13:02.763ZINFOomicron-dev (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
12812025-04-30T06:13:02.763ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = instance_updater elapsed = 2.430999297s iteration = 1 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
12822025-04-30T06:13:02.772ZINFOomicron-dev (dropshot): request completed latency_us = 10025 local_addr = [::1]:61881 method = PUT remote_addr = [::1]:33008 req_id = a8ac3d69-8b35-42b8-994e-003bb175c3f0 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /omicron-config
12832025-04-30T06:13:02.779ZDEBGomicron-dev: client response result = Ok(Response { url: "http://[::1]:61881/omicron-config", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8ac3d69-8b35-42b8-994e-003bb175c3f0", "content-length": "26", "date": "Wed, 30 Apr 2025 06:13:02 GMT"} })
12842025-04-30T06:13:02.789ZTRAComicron-dev (BackgroundTasks): no more instances in need of reincarnation background_task = instance_reincarnation reincarnation_reason = start saga failed total_found = 0
12852025-04-30T06:13:02.790ZINFOomicron-dev (BackgroundTasks): instance reincarnation completed successfully background_task = instance_reincarnation instances_changed_state = 0 instances_found = 0 instances_reincarnated = 0
12862025-04-30T06:13:02.790ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = instance_reincarnation elapsed = 2.457733112s iteration = 1 status = {"changed_state":[],"disabled":false,"errors":[],"instances_found":{"Failed":0,"SagaUnwound":0},"instances_reincarnated":[],"restart_errors":[]}
12872025-04-30T06:13:02.794ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
12882025-04-30T06:13:02.794ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12892025-04-30T06:13:02.794ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12902025-04-30T06:13:02.794ZDEBGomicron-dev: client request body = Some(Body) method = PUT uri = http://[::1]:61881/network-bootstore-config
12912025-04-30T06:13:02.795ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = PUT remote_addr = [::1]:33008 req_id = d9ada273-be2b-45a2-8e70-3dc185412ffe sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
12922025-04-30T06:13:02.795ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12932025-04-30T06:13:02.799ZINFOomicron-dev (dropshot): request completed latency_us = 1813 local_addr = [::1]:61881 method = PUT remote_addr = [::1]:33008 req_id = d9ada273-be2b-45a2-8e70-3dc185412ffe response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /network-bootstore-config
12942025-04-30T06:13:02.799ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
12952025-04-30T06:13:02.799ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
12962025-04-30T06:13:02.799ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
12972025-04-30T06:13:02.799ZDEBGomicron-dev: client response result = Ok(Response { url: "http://[::1]:61881/network-bootstore-config", status: 204, headers: {"x-request-id": "d9ada273-be2b-45a2-8e70-3dc185412ffe", "date": "Wed, 30 Apr 2025 06:13:02 GMT"} })
12982025-04-30T06:13:02.799ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
12992025-04-30T06:13:02.809ZDEBGomicron-dev (init_with_steps): Running step start_nexus_external
13002025-04-30T06:13:02.810ZDEBGomicron-dev: Starting Nexus (external API)
13012025-04-30T06:13:02.854ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13022025-04-30T06:13:02.872ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
13032025-04-30T06:13:02.872ZTRAComicron-dev (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
13042025-04-30T06:13:02.872ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
13052025-04-30T06:13:02.872ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13062025-04-30T06:13:02.872ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13072025-04-30T06:13:02.872ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13082025-04-30T06:13:02.873ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13092025-04-30T06:13:02.877ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13102025-04-30T06:13:02.877ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13112025-04-30T06:13:02.877ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13122025-04-30T06:13:02.878ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13132025-04-30T06:13:02.881ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13142025-04-30T06:13:02.881ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13152025-04-30T06:13:02.881ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13162025-04-30T06:13:02.881ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13172025-04-30T06:13:02.884ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13182025-04-30T06:13:02.891ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
13192025-04-30T06:13:02.891ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13202025-04-30T06:13:02.891ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13212025-04-30T06:13:02.891ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13222025-04-30T06:13:02.912ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13232025-04-30T06:13:02.912ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13242025-04-30T06:13:02.913ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13252025-04-30T06:13:02.921ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13262025-04-30T06:13:02.921ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13272025-04-30T06:13:02.922ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13282025-04-30T06:13:02.950ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13292025-04-30T06:13:02.951ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13302025-04-30T06:13:02.953ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13312025-04-30T06:13:03.004ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:61881/v2p
13322025-04-30T06:13:03.004ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:50619 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13332025-04-30T06:13:03.005ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:50619 req_id = 4f204ffb-8d0a-4186-b658-0cc90792e652 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
13342025-04-30T06:13:03.006ZINFOomicron-dev (dropshot): request completed latency_us = 527 local_addr = [::1]:61881 method = GET remote_addr = [::1]:50619 req_id = 4f204ffb-8d0a-4186-b658-0cc90792e652 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
13352025-04-30T06:13:03.006ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:61881/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f204ffb-8d0a-4186-b658-0cc90792e652", "content-length": "2", "date": "Wed, 30 Apr 2025 06:13:02 GMT"} })
13362025-04-30T06:13:03.010ZINFOomicron-dev (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
13372025-04-30T06:13:03.014ZINFOomicron-dev (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
13382025-04-30T06:13:03.014ZINFOomicron-dev (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
13392025-04-30T06:13:03.014ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 2.674106011s iteration = 1 status = {}
13402025-04-30T06:13:03.032ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13412025-04-30T06:13:03.032ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13422025-04-30T06:13:03.033ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13432025-04-30T06:13:03.065ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
13442025-04-30T06:13:03.065ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13452025-04-30T06:13:03.065ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13462025-04-30T06:13:03.065ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13472025-04-30T06:13:03.066ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13482025-04-30T06:13:03.073ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13492025-04-30T06:13:03.073ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13502025-04-30T06:13:03.074ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13512025-04-30T06:13:03.076ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13522025-04-30T06:13:03.080ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13532025-04-30T06:13:03.080ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13542025-04-30T06:13:03.080ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13552025-04-30T06:13:03.081ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13562025-04-30T06:13:03.095ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13572025-04-30T06:13:03.098ZINFOomicron-dev (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
13582025-04-30T06:13:03.098ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:61881/vpc-routes
13592025-04-30T06:13:03.099ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:34212 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
13602025-04-30T06:13:03.100ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:34212 req_id = 1abdaa24-ae41-4b44-afe3-22b54e06e4d2 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
13612025-04-30T06:13:03.100ZINFOomicron-dev (dropshot): request completed latency_us = 315 local_addr = [::1]:61881 method = GET remote_addr = [::1]:34212 req_id = 1abdaa24-ae41-4b44-afe3-22b54e06e4d2 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
13622025-04-30T06:13:03.100ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:61881/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "1abdaa24-ae41-4b44-afe3-22b54e06e4d2", "content-length": "2", "date": "Wed, 30 Apr 2025 06:13:02 GMT"} })
13632025-04-30T06:13:03.100ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
13642025-04-30T06:13:03.101ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
13652025-04-30T06:13:03.101ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
13662025-04-30T06:13:03.102ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13672025-04-30T06:13:03.102ZTRAComicron-dev (BackgroundTasks): successful lookup of siloed resource "VpcSubnet" using built-in user background_task = service_firewall_rule_propagation
13682025-04-30T06:13:03.102ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
13692025-04-30T06:13:03.102ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
13702025-04-30T06:13:03.102ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13712025-04-30T06:13:03.102ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13722025-04-30T06:13:03.103ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13732025-04-30T06:13:03.120ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
13742025-04-30T06:13:03.121ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13752025-04-30T06:13:03.121ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13762025-04-30T06:13:03.121ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13772025-04-30T06:13:03.125ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13782025-04-30T06:13:03.125ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13792025-04-30T06:13:03.125ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13802025-04-30T06:13:03.127ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13812025-04-30T06:13:03.131ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13822025-04-30T06:13:03.157ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
13832025-04-30T06:13:03.157ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13842025-04-30T06:13:03.158ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13852025-04-30T06:13:03.158ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13862025-04-30T06:13:03.176ZDEBGomicron-dev (service-firewall-rule-progator): resolved names for firewall rules background_task = service_firewall_rule_propagation instance_interfaces = {} subnet_interfaces = {} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
13872025-04-30T06:13:03.177ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet
13882025-04-30T06:13:03.177ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_firewall_rule_propagation resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
13892025-04-30T06:13:03.177ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13902025-04-30T06:13:03.177ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13912025-04-30T06:13:03.179ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13922025-04-30T06:13:03.183ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
13932025-04-30T06:13:03.190ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Fleet result = Ok(())
13942025-04-30T06:13:03.191ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database
13952025-04-30T06:13:03.191ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_firewall_rule_propagation roles = RoleSet { roles: {} }
13962025-04-30T06:13:03.191ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_firewall_rule_propagation resource = Database result = Ok(())
13972025-04-30T06:13:03.227ZTRAComicron-dev (service-firewall-rule-progator): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:06:08.248500Z, allowed_ips: None } background_task = service_firewall_rule_propagation
13982025-04-30T06:13:03.227ZDEBGomicron-dev (service-firewall-rule-progator): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified. background_task = service_firewall_rule_propagation
13992025-04-30T06:13:03.227ZDEBGomicron-dev (service-firewall-rule-progator): resolved firewall rules for sled agents background_task = service_firewall_rule_propagation sled_agent_rules = []
14002025-04-30T06:13:03.227ZDEBGomicron-dev (service-firewall-rule-progator): resolved 0 rules for sleds background_task = service_firewall_rule_propagation
14012025-04-30T06:13:03.281ZINFOomicron-dev (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
14022025-04-30T06:13:03.281ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:61881/eip-gateways
14032025-04-30T06:13:03.282ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = PUT remote_addr = [::1]:34212 req_id = b38332f6-f8f0-4f69-b2c6-460db973500d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
14042025-04-30T06:13:03.282ZINFOomicron-dev (dropshot): request completed latency_us = 507 local_addr = [::1]:61881 method = PUT remote_addr = [::1]:34212 req_id = b38332f6-f8f0-4f69-b2c6-460db973500d response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
14052025-04-30T06:13:03.283ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:61881/eip-gateways", status: 204, headers: {"x-request-id": "b38332f6-f8f0-4f69-b2c6-460db973500d", "date": "Wed, 30 Apr 2025 06:13:03 GMT"} })
14062025-04-30T06:13:03.284ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 2.948847826s iteration = 1 status = {}
14072025-04-30T06:13:03.569ZDEBGomicron-dev (service-firewall-rule-progator): resolved sleds for vpc oxide-services background_task = service_firewall_rule_propagation vpc_to_sled = []
14082025-04-30T06:13:03.569ZDEBGomicron-dev (service-firewall-rule-progator): sending firewall rules to sled agents background_task = service_firewall_rule_propagation
14092025-04-30T06:13:03.569ZINFOomicron-dev (service-firewall-rule-progator): updated firewall rules on 0 sleds background_task = service_firewall_rule_propagation
14102025-04-30T06:13:03.569ZDEBGomicron-dev (service-firewall-rule-progator): successfully propagated service firewall rules background_task = service_firewall_rule_propagation elapsed = 3.235556785s
14112025-04-30T06:13:03.569ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = service_firewall_rule_propagation elapsed = 3.235775401s iteration = 1 status = {}
14122025-04-30T06:13:06.803ZTRAComicron-dev (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
14132025-04-30T06:13:06.803ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14142025-04-30T06:13:06.803ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14152025-04-30T06:13:06.803ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14162025-04-30T06:13:06.803ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14172025-04-30T06:13:06.819ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14182025-04-30T06:13:06.823ZDEBGomicron-dev (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
14192025-04-30T06:13:06.823ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14202025-04-30T06:13:06.823ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14212025-04-30T06:13:06.823ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14222025-04-30T06:13:06.828ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14232025-04-30T06:13:06.828ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14242025-04-30T06:13:06.829ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14252025-04-30T06:13:06.837ZINFOomicron-dev (InternalApi): set cockroachdb setting setting = cluster.preserve_downgrade_option value = 22.1
14262025-04-30T06:13:06.837ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14272025-04-30T06:13:06.837ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14282025-04-30T06:13:06.838ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14292025-04-30T06:13:06.842ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
14302025-04-30T06:13:06.842ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14312025-04-30T06:13:06.842ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14322025-04-30T06:13:06.842ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14332025-04-30T06:13:06.843ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14342025-04-30T06:13:06.846ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14352025-04-30T06:13:06.854ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
14362025-04-30T06:13:06.854ZDEBGomicron-dev (InternalApi): updating rack subnet for rack c19a698f-c6f9-4a17-ae30-20d711b8f7dc to Some( V6( Ipv6Network { addr: fd00:1122:3344:100::, prefix: 56, }, ), )
14372025-04-30T06:13:06.854ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14382025-04-30T06:13:06.854ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14392025-04-30T06:13:06.855ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14402025-04-30T06:13:06.859ZINFOomicron-dev (InternalApi): Using static configuration for external switchports
14412025-04-30T06:13:06.859ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14422025-04-30T06:13:06.859ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14432025-04-30T06:13:06.860ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14442025-04-30T06:13:06.880ZINFOomicron-dev (InternalApi): Recording Rack Network Configuration
14452025-04-30T06:13:06.880ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14462025-04-30T06:13:06.881ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14472025-04-30T06:13:06.882ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14482025-04-30T06:13:06.964ZTRAComicron-dev (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
14492025-04-30T06:13:06.964ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14502025-04-30T06:13:06.964ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14512025-04-30T06:13:06.964ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14522025-04-30T06:13:06.965ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14532025-04-30T06:13:06.968ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14542025-04-30T06:13:06.974ZDEBGomicron-dev (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
14552025-04-30T06:13:06.974ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14562025-04-30T06:13:06.974ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14572025-04-30T06:13:06.976ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14582025-04-30T06:13:06.981ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 2afc7339-274e-4f00-a243-7bdd77fd157f, lookup_type: ByName("oxide-service-pool") }
14592025-04-30T06:13:06.981ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14602025-04-30T06:13:06.981ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14612025-04-30T06:13:06.981ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14622025-04-30T06:13:06.982ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14632025-04-30T06:13:06.986ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14642025-04-30T06:13:06.996ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 2afc7339-274e-4f00-a243-7bdd77fd157f, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14652025-04-30T06:13:06.996ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14662025-04-30T06:13:06.996ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14672025-04-30T06:13:06.996ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14682025-04-30T06:13:07.006ZTRAComicron-dev (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 2afc7339-274e-4f00-a243-7bdd77fd157f, lookup_type: ByName("oxide-service-pool") }
14692025-04-30T06:13:07.006ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14702025-04-30T06:13:07.006ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14712025-04-30T06:13:07.006ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14722025-04-30T06:13:07.007ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14732025-04-30T06:13:07.013ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14742025-04-30T06:13:07.019ZDEBGomicron-dev (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 2afc7339-274e-4f00-a243-7bdd77fd157f, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14752025-04-30T06:13:07.034ZTRAComicron-dev (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 2afc7339-274e-4f00-a243-7bdd77fd157f, lookup_type: ByName("oxide-service-pool") }
14762025-04-30T06:13:07.034ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14772025-04-30T06:13:07.034ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14782025-04-30T06:13:07.035ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14792025-04-30T06:13:07.035ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14802025-04-30T06:13:07.038ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14812025-04-30T06:13:07.043ZDEBGomicron-dev (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = IpPool { parent: Fleet, key: 2afc7339-274e-4f00-a243-7bdd77fd157f, lookup_type: ByName("oxide-service-pool") } result = Ok(())
14822025-04-30T06:13:07.048ZTRAComicron-dev (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
14832025-04-30T06:13:07.048ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14842025-04-30T06:13:07.048ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14852025-04-30T06:13:07.048ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14862025-04-30T06:13:07.049ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14872025-04-30T06:13:07.052ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14882025-04-30T06:13:07.058ZDEBGomicron-dev (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
14892025-04-30T06:13:07.225ZINFOomicron-dev (InternalApi): inserted blueprint blueprint_id = 1bfc4639-5ca6-4323-a678-d4a0347a6fdf
14902025-04-30T06:13:07.225ZTRAComicron-dev (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig
14912025-04-30T06:13:07.225ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
14922025-04-30T06:13:07.225ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
14932025-04-30T06:13:07.225ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
14942025-04-30T06:13:07.226ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
14952025-04-30T06:13:07.229ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
14962025-04-30T06:13:07.234ZDEBGomicron-dev (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = BlueprintConfig result = Ok(())
14972025-04-30T06:13:07.247ZINFOomicron-dev (InternalApi): No networking records needed for clickhouse service
14982025-04-30T06:13:07.247ZINFOomicron-dev (InternalApi): No networking records needed for crucible_pantry service
14992025-04-30T06:13:07.305ZINFOomicron-dev (InternalApi): Inserted networking records for external_dns service
15002025-04-30T06:13:07.305ZINFOomicron-dev (InternalApi): No networking records needed for cockroach_db service
15012025-04-30T06:13:07.305ZINFOomicron-dev (InternalApi): No networking records needed for internal_dns service
15022025-04-30T06:13:07.333ZINFOomicron-dev (InternalApi): Inserted networking records for nexus service
15032025-04-30T06:13:07.333ZINFOomicron-dev (InternalApi): Inserted service networking records
15042025-04-30T06:13:07.333ZINFOomicron-dev (InternalApi): Inserted physical disks
15052025-04-30T06:13:07.333ZINFOomicron-dev (InternalApi): Inserted zpools
15062025-04-30T06:13:07.333ZINFOomicron-dev (InternalApi): Inserted datasets
15072025-04-30T06:13:07.349ZINFOomicron-dev (InternalApi): Populated DNS tables for internal DNS
15082025-04-30T06:13:07.358ZINFOomicron-dev (InternalApi): Populated DNS tables for external DNS
15092025-04-30T06:13:07.361ZTRAComicron-dev (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15102025-04-30T06:13:07.361ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15112025-04-30T06:13:07.361ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15122025-04-30T06:13:07.361ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15132025-04-30T06:13:07.362ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15142025-04-30T06:13:07.369ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15152025-04-30T06:13:07.373ZDEBGomicron-dev (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15162025-04-30T06:13:07.373ZTRAComicron-dev (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15172025-04-30T06:13:07.373ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15182025-04-30T06:13:07.373ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15192025-04-30T06:13:07.373ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15202025-04-30T06:13:07.374ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15212025-04-30T06:13:07.380ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15222025-04-30T06:13:07.385ZDEBGomicron-dev (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15232025-04-30T06:13:07.429ZTRAComicron-dev (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
15242025-04-30T06:13:07.429ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15252025-04-30T06:13:07.429ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15262025-04-30T06:13:07.429ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15272025-04-30T06:13:07.430ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15282025-04-30T06:13:07.434ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15292025-04-30T06:13:07.448ZDEBGomicron-dev (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
15302025-04-30T06:13:07.448ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
15312025-04-30T06:13:07.448ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15322025-04-30T06:13:07.448ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15332025-04-30T06:13:07.448ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15342025-04-30T06:13:07.449ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15352025-04-30T06:13:07.452ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15362025-04-30T06:13:07.458ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
15372025-04-30T06:13:07.464ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig
15382025-04-30T06:13:07.464ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15392025-04-30T06:13:07.464ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15402025-04-30T06:13:07.464ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15412025-04-30T06:13:07.465ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15422025-04-30T06:13:07.476ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15432025-04-30T06:13:07.488ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = DnsConfig result = Ok(())
15442025-04-30T06:13:07.500ZINFOomicron-dev (InternalApi): Created recovery silo
15452025-04-30T06:13:07.502ZINFOomicron-dev (InternalApi): Created recovery user
15462025-04-30T06:13:07.510ZINFOomicron-dev (InternalApi): Created recovery user's password
15472025-04-30T06:13:07.510ZTRAComicron-dev (InternalApi): authorize begin action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 5f8b1342-cf5b-46a4-a95c-4e40f3bc0974, lookup_type: ById(5f8b1342-cf5b-46a4-a95c-4e40f3bc0974) }
15482025-04-30T06:13:07.510ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 5f8b1342-cf5b-46a4-a95c-4e40f3bc0974 resource_type = Silo
15492025-04-30T06:13:07.510ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15502025-04-30T06:13:07.510ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15512025-04-30T06:13:07.511ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15522025-04-30T06:13:07.514ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15532025-04-30T06:13:07.514ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15542025-04-30T06:13:07.514ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15552025-04-30T06:13:07.514ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15562025-04-30T06:13:07.517ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15572025-04-30T06:13:07.520ZDEBGomicron-dev (InternalApi): authorize result action = ModifyPolicy actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Silo { parent: Fleet, key: 5f8b1342-cf5b-46a4-a95c-4e40f3bc0974, lookup_type: ById(5f8b1342-cf5b-46a4-a95c-4e40f3bc0974) } result = Ok(())
15582025-04-30T06:13:07.520ZDEBGomicron-dev (InternalApi): Generated role assignment queries
15592025-04-30T06:13:07.525ZINFOomicron-dev (InternalApi): Granted Silo privileges
15602025-04-30T06:13:07.525ZTRAComicron-dev (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
15612025-04-30T06:13:07.526ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15622025-04-30T06:13:07.526ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15632025-04-30T06:13:07.526ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15642025-04-30T06:13:07.526ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15652025-04-30T06:13:07.529ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15662025-04-30T06:13:07.532ZDEBGomicron-dev (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
15672025-04-30T06:13:07.543ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15682025-04-30T06:13:07.543ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15692025-04-30T06:13:07.544ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15702025-04-30T06:13:07.546ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15712025-04-30T06:13:07.546ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15722025-04-30T06:13:07.547ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15732025-04-30T06:13:07.549ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15742025-04-30T06:13:07.549ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15752025-04-30T06:13:07.550ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15762025-04-30T06:13:07.553ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
15772025-04-30T06:13:07.553ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15782025-04-30T06:13:07.553ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15792025-04-30T06:13:07.553ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15802025-04-30T06:13:07.554ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15812025-04-30T06:13:07.556ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15822025-04-30T06:13:07.557ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15832025-04-30T06:13:07.557ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15842025-04-30T06:13:07.557ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15852025-04-30T06:13:07.561ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
15862025-04-30T06:13:07.561ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15872025-04-30T06:13:07.561ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15882025-04-30T06:13:07.562ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15892025-04-30T06:13:07.565ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
15902025-04-30T06:13:07.572ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
15912025-04-30T06:13:07.572ZTRAComicron-dev (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
15922025-04-30T06:13:07.572ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
15932025-04-30T06:13:07.572ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
15942025-04-30T06:13:07.572ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15952025-04-30T06:13:07.572ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
15962025-04-30T06:13:07.573ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
15972025-04-30T06:13:07.575ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
15982025-04-30T06:13:07.575ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
15992025-04-30T06:13:07.575ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16002025-04-30T06:13:07.576ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16012025-04-30T06:13:07.579ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16022025-04-30T06:13:07.579ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16032025-04-30T06:13:07.579ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16042025-04-30T06:13:07.579ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16052025-04-30T06:13:07.582ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16062025-04-30T06:13:07.597ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16072025-04-30T06:13:07.598ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16082025-04-30T06:13:07.598ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16092025-04-30T06:13:07.598ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16102025-04-30T06:13:07.603ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16112025-04-30T06:13:07.603ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16122025-04-30T06:13:07.604ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16132025-04-30T06:13:07.607ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16142025-04-30T06:13:07.607ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16152025-04-30T06:13:07.609ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16162025-04-30T06:13:07.613ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16172025-04-30T06:13:07.613ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16182025-04-30T06:13:07.614ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16192025-04-30T06:13:07.617ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
16202025-04-30T06:13:07.617ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16212025-04-30T06:13:07.617ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16222025-04-30T06:13:07.617ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16232025-04-30T06:13:07.618ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16242025-04-30T06:13:07.622ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16252025-04-30T06:13:07.622ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16262025-04-30T06:13:07.622ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16272025-04-30T06:13:07.623ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16282025-04-30T06:13:07.627ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16292025-04-30T06:13:07.627ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16302025-04-30T06:13:07.627ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16312025-04-30T06:13:07.629ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16322025-04-30T06:13:07.634ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16332025-04-30T06:13:07.645ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
16342025-04-30T06:13:07.645ZTRAComicron-dev (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
16352025-04-30T06:13:07.645ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16362025-04-30T06:13:07.645ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16372025-04-30T06:13:07.646ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16382025-04-30T06:13:07.651ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16392025-04-30T06:13:07.651ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16402025-04-30T06:13:07.652ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16412025-04-30T06:13:07.655ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16422025-04-30T06:13:07.655ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16432025-04-30T06:13:07.656ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16442025-04-30T06:13:07.661ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16452025-04-30T06:13:07.661ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16462025-04-30T06:13:07.662ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16472025-04-30T06:13:07.664ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
16482025-04-30T06:13:07.664ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16492025-04-30T06:13:07.665ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16502025-04-30T06:13:07.665ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16512025-04-30T06:13:07.665ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16522025-04-30T06:13:07.667ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16532025-04-30T06:13:07.667ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16542025-04-30T06:13:07.667ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16552025-04-30T06:13:07.668ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16562025-04-30T06:13:07.669ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16572025-04-30T06:13:07.669ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16582025-04-30T06:13:07.669ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16592025-04-30T06:13:07.670ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16602025-04-30T06:13:07.673ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16612025-04-30T06:13:07.680ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16622025-04-30T06:13:07.681ZTRAComicron-dev (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
16632025-04-30T06:13:07.681ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
16642025-04-30T06:13:07.681ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16652025-04-30T06:13:07.681ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16662025-04-30T06:13:07.681ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16672025-04-30T06:13:07.681ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16682025-04-30T06:13:07.684ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16692025-04-30T06:13:07.684ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16702025-04-30T06:13:07.684ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16712025-04-30T06:13:07.685ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16722025-04-30T06:13:07.688ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
16732025-04-30T06:13:07.688ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16742025-04-30T06:13:07.689ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16752025-04-30T06:13:07.691ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16762025-04-30T06:13:07.693ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
16772025-04-30T06:13:07.709ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
16782025-04-30T06:13:07.709ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16792025-04-30T06:13:07.709ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16802025-04-30T06:13:07.710ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16812025-04-30T06:13:07.719ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16822025-04-30T06:13:07.719ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16832025-04-30T06:13:07.720ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16842025-04-30T06:13:07.724ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16852025-04-30T06:13:07.724ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16862025-04-30T06:13:07.731ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16872025-04-30T06:13:07.735ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16882025-04-30T06:13:07.735ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16892025-04-30T06:13:07.736ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16902025-04-30T06:13:07.738ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16912025-04-30T06:13:07.738ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16922025-04-30T06:13:07.739ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16932025-04-30T06:13:07.742ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
16942025-04-30T06:13:07.742ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
16952025-04-30T06:13:07.742ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
16962025-04-30T06:13:07.742ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
16972025-04-30T06:13:07.743ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
16982025-04-30T06:13:07.745ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
16992025-04-30T06:13:07.745ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17002025-04-30T06:13:07.745ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
17012025-04-30T06:13:07.746ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17022025-04-30T06:13:07.749ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17032025-04-30T06:13:07.749ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17042025-04-30T06:13:07.749ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
17052025-04-30T06:13:07.750ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17062025-04-30T06:13:07.752ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17072025-04-30T06:13:07.764ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17082025-04-30T06:13:07.764ZTRAComicron-dev (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
17092025-04-30T06:13:07.765ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
17102025-04-30T06:13:07.765ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
17112025-04-30T06:13:07.765ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17122025-04-30T06:13:07.765ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
17132025-04-30T06:13:07.765ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17142025-04-30T06:13:07.768ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
17152025-04-30T06:13:07.768ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17162025-04-30T06:13:07.768ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
17172025-04-30T06:13:07.771ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17182025-04-30T06:13:07.774ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17192025-04-30T06:13:07.774ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17202025-04-30T06:13:07.774ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
17212025-04-30T06:13:07.775ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17222025-04-30T06:13:07.780ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17232025-04-30T06:13:07.794ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
17242025-04-30T06:13:07.794ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17252025-04-30T06:13:07.794ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
17262025-04-30T06:13:07.795ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17272025-04-30T06:13:07.807ZDEBGomicron-dev (nexus): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("nexus"): [NetworkInterface { id: cf66f81f-75b3-463a-96ab-2fe8219455e6, kind: Service { id: e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c }, name: Name("nexus-e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("external-dns"): [NetworkInterface { id: 925b1d1e-94b1-469c-94f6-b225eb8d1c11, kind: Service { id: 90372d2a-e95e-4302-b338-2843dd07d33a }, name: Name("external-dns-90372d2a-e95e-4302-b338-2843dd07d33a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
17282025-04-30T06:13:07.808ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
17292025-04-30T06:13:07.808ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17302025-04-30T06:13:07.808ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17312025-04-30T06:13:07.809ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
17322025-04-30T06:13:07.810ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17332025-04-30T06:13:07.816ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
17342025-04-30T06:13:07.828ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
17352025-04-30T06:13:07.828ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
17362025-04-30T06:13:07.828ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
17372025-04-30T06:13:07.829ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
17382025-04-30T06:13:07.832ZTRAComicron-dev (nexus): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:13:07.532584Z, allowed_ips: None }
17392025-04-30T06:13:07.832ZDEBGomicron-dev (nexus): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
17402025-04-30T06:13:07.842ZDEBGomicron-dev (nexus): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 925b1d1e-94b1-469c-94f6-b225eb8d1c11, kind: Service { id: 90372d2a-e95e-4302-b338-2843dd07d33a }, name: Name("external-dns-90372d2a-e95e-4302-b338-2843dd07d33a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: cf66f81f-75b3-463a-96ab-2fe8219455e6, kind: Service { id: e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c }, name: Name("nexus-e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
17412025-04-30T06:13:07.842ZDEBGomicron-dev (nexus): resolved 2 rules for sleds
17422025-04-30T06:13:07.939ZDEBGomicron-dev (nexus): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2025-04-30T06:13:01.188956Z, time_modified: 2025-04-30T06:13:01.188956Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(61881), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(43978) }]
17432025-04-30T06:13:07.941ZDEBGomicron-dev (nexus): sending firewall rules to sled agents
17442025-04-30T06:13:07.941ZTRAComicron-dev (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
17452025-04-30T06:13:07.941ZDEBGomicron-dev (InstanceAllocator): roles roles = RoleSet { roles: {} }
17462025-04-30T06:13:07.942ZDEBGomicron-dev (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
17472025-04-30T06:13:07.947ZTRAComicron-dev (InstanceAllocator): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
17482025-04-30T06:13:07.947ZTRAComicron-dev (InstanceAllocator): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17492025-04-30T06:13:07.947ZTRAComicron-dev (InstanceAllocator): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database
17502025-04-30T06:13:07.947ZDEBGomicron-dev (InstanceAllocator): roles roles = RoleSet { roles: {} }
17512025-04-30T06:13:07.948ZDEBGomicron-dev (InstanceAllocator): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Database result = Ok(())
17522025-04-30T06:13:07.951ZDEBGomicron-dev (InstanceAllocator): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "viewer")} }
17532025-04-30T06:13:07.954ZDEBGomicron-dev (InstanceAllocator): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000004ead, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
17542025-04-30T06:13:07.992ZDEBGomicron-dev (nexus): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:61881/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17552025-04-30T06:13:07.993ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:65309 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
17562025-04-30T06:13:07.994ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = PUT remote_addr = [::1]:65309 req_id = 3a0632cb-0dd9-4316-acdb-1ebea6a2fff0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17572025-04-30T06:13:07.996ZINFOomicron-dev (dropshot): request completed latency_us = 2036 local_addr = [::1]:61881 method = PUT remote_addr = [::1]:65309 req_id = 3a0632cb-0dd9-4316-acdb-1ebea6a2fff0 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
17582025-04-30T06:13:07.996ZDEBGomicron-dev (nexus): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:61881/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "3a0632cb-0dd9-4316-acdb-1ebea6a2fff0", "date": "Wed, 30 Apr 2025 06:13:07 GMT"} })
17592025-04-30T06:13:07.997ZINFOomicron-dev (nexus): updated firewall rules on 1 sleds
17602025-04-30T06:13:07.997ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_config_internal dns_group = internal iteration = 2 reason = Signaled
17612025-04-30T06:13:07.997ZDEBGomicron-dev (BackgroundTasks): reading DNS version background_task = dns_config_internal dns_group = internal
17622025-04-30T06:13:07.997ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
17632025-04-30T06:13:07.997ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
17642025-04-30T06:13:07.997ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_servers_internal dns_group = internal iteration = 2 reason = Signaled
17652025-04-30T06:13:07.997ZTRAComicron-dev (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
17662025-04-30T06:13:07.997ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal
17672025-04-30T06:13:07.997ZDEBGomicron-dev (ServiceBalancer): roles roles = RoleSet { roles: {} }
17682025-04-30T06:13:07.997ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_config_external dns_group = external iteration = 2 reason = Signaled
17692025-04-30T06:13:07.997ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_servers_external dns_group = external iteration = 2 reason = Signaled
17702025-04-30T06:13:07.997ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal
17712025-04-30T06:13:07.998ZDEBGomicron-dev (BackgroundTasks): reading DNS version background_task = dns_config_external dns_group = external
17722025-04-30T06:13:07.998ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
17732025-04-30T06:13:07.998ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
17742025-04-30T06:13:07.998ZDEBGomicron-dev (BackgroundTasks): activating background_task = external_endpoints iteration = 2 reason = Signaled
17752025-04-30T06:13:07.998ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 3 reason = Signaled
17762025-04-30T06:13:07.998ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
17772025-04-30T06:13:07.998ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
17782025-04-30T06:13:07.998ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17792025-04-30T06:13:07.998ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
17802025-04-30T06:13:07.998ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
17812025-04-30T06:13:07.998ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
17822025-04-30T06:13:07.998ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
17832025-04-30T06:13:07.998ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
17842025-04-30T06:13:07.998ZTRAComicron-dev (dns): buffer buffer = Length: 76 (0x4c) bytes\n0000: 74 24 01 00 00 01 00 00 00 00 00 01 0d 5f 65 78 t$..........._ex\n0010: 74 65 72 6e 61 6c 2d 64 6e 73 04 5f 74 63 70 0d ternal-dns._tcp.\n0020: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0030: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 ide.internal..!.\n0040: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64817 req_id = e060f22b-6080-44bb-9fd0-f54d6090a28a
17852025-04-30T06:13:07.998ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29732,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_external-dns._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_external-dns._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 13,\n 95,\n 101,\n 120,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 45,\n 100,\n 110,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64817 req_id = e060f22b-6080-44bb-9fd0-f54d6090a28a
17862025-04-30T06:13:07.998ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
17872025-04-30T06:13:07.998ZTRAComicron-dev (dns): buffer buffer = Length: 75 (0x4b) bytes\n0000: de 01 01 00 00 01 00 00 00 00 00 01 0c 5f 6e 61 ............._na\n0010: 6d 65 73 65 72 76 69 63 65 04 5f 74 63 70 0d 63 meservice._tcp.c\n0020: 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 ontrol-plane.oxi\n0030: 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 de.internal..!..\n0040: 00 00 29 04 d0 00 00 00 00 00 00 ..)........ kind = dns peer_addr = [::1]:52634 req_id = 150ac5ad-6a9f-4f7d-a953-09e47a1a9f5d
17882025-04-30T06:13:07.998ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56833,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_nameservice._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_nameservice._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 12,\n 95,\n 110,\n 97,\n 109,\n 101,\n 115,\n 101,\n 114,\n 118,\n 105,\n 99,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52634 req_id = 150ac5ad-6a9f-4f7d-a953-09e47a1a9f5d
17892025-04-30T06:13:07.998ZDEBGomicron-dev (store): query key key = _external-dns._tcp
17902025-04-30T06:13:07.998ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
17912025-04-30T06:13:07.998ZDEBGomicron-dev (store): query key key = _nameservice._tcp
17922025-04-30T06:13:07.998ZDEBGomicron-dev (store): query key key = cd92814a-77d2-4d97-8e60-d67fa56f9567.host
17932025-04-30T06:13:07.998ZDEBGomicron-dev (store): query key key = 90372d2a-e95e-4302-b338-2843dd07d33a.host
17942025-04-30T06:13:07.998ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
17952025-04-30T06:13:07.999ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
17962025-04-30T06:13:07.999ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64817 query = LowerQuery { name: LowerName(Name("_external-dns._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 54696, target: Name("90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal") })) }] req_id = e060f22b-6080-44bb-9fd0-f54d6090a28a
17972025-04-30T06:13:07.999ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52634 query = LowerQuery { name: LowerName(Name("_nameservice._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39475, target: Name("cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal") })) }] req_id = 150ac5ad-6a9f-4f7d-a953-09e47a1a9f5d
17982025-04-30T06:13:07.999ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _external-dns._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_external-dns._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_external-dns._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 54696, target: Name("90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2452, tv_nsec: 798414597 } })
17992025-04-30T06:13:07.999ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _nameservice._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_nameservice._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_nameservice._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 39475, target: Name("cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2452, tv_nsec: 798446921 } })
18002025-04-30T06:13:07.999ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
18012025-04-30T06:13:07.999ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
18022025-04-30T06:13:07.999ZDEBGomicron-dev (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
18032025-04-30T06:13:07.999ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
18042025-04-30T06:13:07.999ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18052025-04-30T06:13:07.999ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
18062025-04-30T06:13:07.999ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18072025-04-30T06:13:08.001ZTRAComicron-dev (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 9a fb 01 00 00 01 00 00 00 00 00 01 24 39 30 33 ............$903\n0010: 37 32 64 32 61 2d 65 39 35 65 2d 34 33 30 32 2d 72d2a-e95e-4302-\n0020: 62 33 33 38 2d 32 38 34 33 64 64 30 37 64 33 33 b338-2843dd07d33\n0030: 61 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 a.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:52100 req_id = 158ec307-3034-421c-a7e4-c0368e0de705
18082025-04-30T06:13:08.001ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39675,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 57,\n 48,\n 51,\n 55,\n 50,\n 100,\n 50,\n 97,\n 45,\n 101,\n 57,\n 53,\n 101,\n 45,\n 52,\n 51,\n 48,\n 50,\n 45,\n 98,\n 51,\n 51,\n 56,\n 45,\n 50,\n 56,\n 52,\n 51,\n 100,\n 100,\n 48,\n 55,\n 100,\n 51,\n 51,\n 97,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52100 req_id = 158ec307-3034-421c-a7e4-c0368e0de705
18092025-04-30T06:13:08.001ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18102025-04-30T06:13:08.001ZDEBGomicron-dev (store): query key key = 90372d2a-e95e-4302-b338-2843dd07d33a.host
18112025-04-30T06:13:08.001ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52100 query = LowerQuery { name: LowerName(Name("90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal.")), original: Query { name: Name("90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("90372d2a-e95e-4302-b338-2843dd07d33a.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 158ec307-3034-421c-a7e4-c0368e0de705
18122025-04-30T06:13:08.001ZTRAComicron-dev (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: 68 75 01 00 00 01 00 00 00 00 00 01 24 63 64 39 hu..........$cd9\n0010: 32 38 31 34 61 2d 37 37 64 32 2d 34 64 39 37 2d 2814a-77d2-4d97-\n0020: 38 65 36 30 2d 64 36 37 66 61 35 36 66 39 35 36 8e60-d67fa56f956\n0030: 37 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 7.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:58309 req_id = 01792403-a28e-49fb-888a-5ac4611d7de6
18132025-04-30T06:13:08.001ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26741,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 99,\n 100,\n 57,\n 50,\n 56,\n 49,\n 52,\n 97,\n 45,\n 55,\n 55,\n 100,\n 50,\n 45,\n 52,\n 100,\n 57,\n 55,\n 45,\n 56,\n 101,\n 54,\n 48,\n 45,\n 100,\n 54,\n 55,\n 102,\n 97,\n 53,\n 54,\n 102,\n 57,\n 53,\n 54,\n 55,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58309 req_id = 01792403-a28e-49fb-888a-5ac4611d7de6
18142025-04-30T06:13:08.001ZINFOomicron-dev (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:54696] } background_task = dns_servers_external dns_group = external
18152025-04-30T06:13:08.001ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_servers_external dns_group = external elapsed = 3.640593ms iteration = 2 status = {"addresses":["[::1]:54696"]}
18162025-04-30T06:13:08.001ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 2 reason = Dependency
18172025-04-30T06:13:08.001ZWARNomicron-dev (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_external dns_group = external reason = no config
18182025-04-30T06:13:08.001ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 33.647\u{b5}s iteration = 2 status = {"error":"no config"}
18192025-04-30T06:13:08.001ZDEBGomicron-dev (store): query key key = cd92814a-77d2-4d97-8e60-d67fa56f9567.host
18202025-04-30T06:13:08.001ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58309 query = LowerQuery { name: LowerName(Name("cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal.")), original: Query { name: Name("cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("cd92814a-77d2-4d97-8e60-d67fa56f9567.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 01792403-a28e-49fb-888a-5ac4611d7de6
18212025-04-30T06:13:08.001ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
18222025-04-30T06:13:08.001ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18232025-04-30T06:13:08.001ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
18242025-04-30T06:13:08.001ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
18252025-04-30T06:13:08.002ZINFOomicron-dev (BackgroundTasks): found DNS servers (initial) addresses = DnsServersList { addresses: [[::1]:39475] } background_task = dns_servers_internal dns_group = internal
18262025-04-30T06:13:08.002ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_servers_internal dns_group = internal elapsed = 4.306901ms iteration = 2 status = {"addresses":["[::1]:39475"]}
18272025-04-30T06:13:08.002ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 2 reason = Dependency
18282025-04-30T06:13:08.002ZWARNomicron-dev (BackgroundTasks): DNS propagation: skipped background_task = dns_propagation_internal dns_group = internal reason = no config
18292025-04-30T06:13:08.002ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 22.034\u{b5}s iteration = 2 status = {"error":"no config"}
18302025-04-30T06:13:08.002ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
18312025-04-30T06:13:08.005ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18322025-04-30T06:13:08.005ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18332025-04-30T06:13:08.007ZTRAComicron-dev (ServiceBalancer): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) }
18342025-04-30T06:13:08.007ZTRAComicron-dev (ServiceBalancer): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18352025-04-30T06:13:08.007ZTRAComicron-dev (ServiceBalancer): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database
18362025-04-30T06:13:08.007ZDEBGomicron-dev (ServiceBalancer): roles roles = RoleSet { roles: {} }
18372025-04-30T06:13:08.008ZDEBGomicron-dev (ServiceBalancer): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Database result = Ok(())
18382025-04-30T06:13:08.010ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
18392025-04-30T06:13:08.010ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
18402025-04-30T06:13:08.010ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18412025-04-30T06:13:08.010ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
18422025-04-30T06:13:08.010ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
18432025-04-30T06:13:08.010ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
18442025-04-30T06:13:08.010ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18452025-04-30T06:13:08.010ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
18462025-04-30T06:13:08.012ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18472025-04-30T06:13:08.013ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18482025-04-30T06:13:08.014ZDEBGomicron-dev (ServiceBalancer): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18492025-04-30T06:13:08.018ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18502025-04-30T06:13:08.020ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
18512025-04-30T06:13:08.020ZDEBGomicron-dev (BackgroundTasks): found DNS version background_task = dns_config_internal dns_group = internal version = 1
18522025-04-30T06:13:08.020ZDEBGomicron-dev (BackgroundTasks): reading DNS config background_task = dns_config_internal dns_group = internal
18532025-04-30T06:13:08.020ZDEBGomicron-dev (BackgroundTasks): reading DNS zones background_task = dns_config_internal dns_group = internal
18542025-04-30T06:13:08.020ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
18552025-04-30T06:13:08.020ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18562025-04-30T06:13:08.020ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
18572025-04-30T06:13:08.020ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18582025-04-30T06:13:08.021ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18592025-04-30T06:13:08.021ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig
18602025-04-30T06:13:08.021ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18612025-04-30T06:13:08.021ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
18622025-04-30T06:13:08.021ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18632025-04-30T06:13:08.022ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18642025-04-30T06:13:08.024ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = []
18652025-04-30T06:13:08.024ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
18662025-04-30T06:13:08.024ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18672025-04-30T06:13:08.024ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18682025-04-30T06:13:08.025ZDEBGomicron-dev (ServiceBalancer): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-00000000bac3, .. }) resource = Rack { parent: Fleet, key: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, lookup_type: ById(c19a698f-c6f9-4a17-ae30-20d711b8f7dc) } result = Ok(())
18692025-04-30T06:13:08.025ZINFOomicron-dev (nexus): Rack initialized
18702025-04-30T06:13:08.025ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18712025-04-30T06:13:08.025ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 51 0e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 Q............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60448 req_id = 9cb824d7-e6c3-408b-af72-47c809d39b52
18722025-04-30T06:13:08.025ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
18732025-04-30T06:13:08.025ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20750,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60448 req_id = 9cb824d7-e6c3-408b-af72-47c809d39b52
18742025-04-30T06:13:08.025ZDEBGomicron-dev (store): query key key = _mgs._tcp
18752025-04-30T06:13:08.026ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
18762025-04-30T06:13:08.026ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60448 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9cb824d7-e6c3-408b-af72-47c809d39b52
18772025-04-30T06:13:08.026ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2452, tv_nsec: 825456997 } })
18782025-04-30T06:13:08.026ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
18792025-04-30T06:13:08.026ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
18802025-04-30T06:13:08.027ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 20 67 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e g..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57532 req_id = 411d8bc7-2528-4218-84d2-2d3340bded12
18812025-04-30T06:13:08.027ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8295,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57532 req_id = 411d8bc7-2528-4218-84d2-2d3340bded12
18822025-04-30T06:13:08.027ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
18832025-04-30T06:13:08.027ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57532 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 411d8bc7-2528-4218-84d2-2d3340bded12
18842025-04-30T06:13:08.030ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = DnsConfig result = Ok(())
18852025-04-30T06:13:08.031ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
18862025-04-30T06:13:08.031ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18872025-04-30T06:13:08.031ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
18882025-04-30T06:13:08.031ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
18892025-04-30T06:13:08.031ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
18902025-04-30T06:13:08.031ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18912025-04-30T06:13:08.031ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
18922025-04-30T06:13:08.036ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet
18932025-04-30T06:13:08.036ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = external_endpoints resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
18942025-04-30T06:13:08.036ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
18952025-04-30T06:13:08.036ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
18962025-04-30T06:13:08.036ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
18972025-04-30T06:13:08.039ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
18982025-04-30T06:13:08.040ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
18992025-04-30T06:13:08.040ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
19002025-04-30T06:13:08.043ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Fleet result = Ok(())
19012025-04-30T06:13:08.043ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database
19022025-04-30T06:13:08.043ZDEBGomicron-dev (BackgroundTasks): roles background_task = external_endpoints roles = RoleSet { roles: {} }
19032025-04-30T06:13:08.043ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19042025-04-30T06:13:08.043ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = external_endpoints resource = Database result = Ok(())
19052025-04-30T06:13:08.050ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19062025-04-30T06:13:08.055ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
19072025-04-30T06:13:08.055ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19082025-04-30T06:13:08.064ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
19092025-04-30T06:13:08.065ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 49 be 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c I............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:49861 req_id = 393eccdc-3eef-43f7-bf17-6a77a8a7f8a7
19102025-04-30T06:13:08.065ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 18878,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49861 req_id = 393eccdc-3eef-43f7-bf17-6a77a8a7f8a7
19112025-04-30T06:13:08.065ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
19122025-04-30T06:13:08.067ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18878, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49861 req_id = 393eccdc-3eef-43f7-bf17-6a77a8a7f8a7
19132025-04-30T06:13:08.068ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
19142025-04-30T06:13:08.068ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
19152025-04-30T06:13:08.068ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
19162025-04-30T06:13:08.070ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "32afbdd8-2355-470a-99a5-f9ef9600cb69", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19172025-04-30T06:13:08.073ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
19182025-04-30T06:13:08.074ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
19192025-04-30T06:13:08.074ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19202025-04-30T06:13:08.075ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
19212025-04-30T06:13:08.081ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "84b39798-9e59-4358-838a-9ec0adcd4591", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19222025-04-30T06:13:08.082ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
19232025-04-30T06:13:08.083ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19242025-04-30T06:13:08.084ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
19252025-04-30T06:13:08.093ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21a88440-0955-4646-8a4b-a181e682f9a2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:07 GMT"} })
19262025-04-30T06:13:08.093ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
19272025-04-30T06:13:08.096ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9081dfe5-d8d0-4619-aaa3-b0af3862250b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19282025-04-30T06:13:08.096ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
19292025-04-30T06:13:08.099ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebcd9eea-9ab3-44fd-ab35-ed3acbbaec3c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19302025-04-30T06:13:08.099ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
19312025-04-30T06:13:08.101ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3a795ab-3a2e-4817-8224-dff0545af4ea", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19322025-04-30T06:13:08.102ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
19332025-04-30T06:13:08.104ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e93ae5a-87a0-4d97-aa50-a88ea2ad15e0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19342025-04-30T06:13:08.104ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
19352025-04-30T06:13:08.107ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "20045311-3f73-44ac-b892-7ad64dffb87e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19362025-04-30T06:13:08.109ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
19372025-04-30T06:13:08.109ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19382025-04-30T06:13:08.109ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
19392025-04-30T06:13:08.110ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19402025-04-30T06:13:08.113ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19412025-04-30T06:13:08.113ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19422025-04-30T06:13:08.114ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
19432025-04-30T06:13:08.114ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19442025-04-30T06:13:08.118ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19452025-04-30T06:13:08.126ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
19462025-04-30T06:13:08.126ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
19472025-04-30T06:13:08.126ZTRAComicron-dev (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
19482025-04-30T06:13:08.126ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
19492025-04-30T06:13:08.126ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
19502025-04-30T06:13:08.126ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19512025-04-30T06:13:08.126ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
19522025-04-30T06:13:08.126ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19532025-04-30T06:13:08.129ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d111982-8bd3-46dd-8dec-d350a2b1d8d3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:07 GMT"} })
19542025-04-30T06:13:08.129ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
19552025-04-30T06:13:08.129ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19562025-04-30T06:13:08.129ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
19572025-04-30T06:13:08.129ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
19582025-04-30T06:13:08.135ZDEBGomicron-dev (BackgroundTasks): found zones background_task = dns_config_internal count = 1 dns_group = internal
19592025-04-30T06:13:08.135ZDEBGomicron-dev (BackgroundTasks): listing DNS names for zone background_task = dns_config_internal batch_size = 100 dns_group = internal dns_zone_id = e4b87b6b-5de9-4b2c-9d61-c510323fdb21 dns_zone_name = control-plane.oxide.internal found_so_far = 0 version = 1
19602025-04-30T06:13:08.135ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig
19612025-04-30T06:13:08.135ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_internal dns_group = internal resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19622025-04-30T06:13:08.135ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19632025-04-30T06:13:08.135ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19642025-04-30T06:13:08.136ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "13b5792b-114b-4b4a-9f05-a278ab87f5d2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19652025-04-30T06:13:08.136ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19662025-04-30T06:13:08.136ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19672025-04-30T06:13:08.137ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
19682025-04-30T06:13:08.140ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf3e223c-a369-4552-836a-9f6e3e8eaa1d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:07 GMT"} })
19692025-04-30T06:13:08.140ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
19702025-04-30T06:13:08.141ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19712025-04-30T06:13:08.142ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9042ab5-3429-44ac-8413-1c6bdb5901c5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19722025-04-30T06:13:08.142ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
19732025-04-30T06:13:08.145ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "11e88446-2907-4d3a-a2ec-70fd1d9f0b20", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:07 GMT"} })
19742025-04-30T06:13:08.145ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
19752025-04-30T06:13:08.146ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19762025-04-30T06:13:08.149ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
19772025-04-30T06:13:08.149ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
19782025-04-30T06:13:08.151ZDEBGomicron-dev (BackgroundTasks): found DNS version background_task = dns_config_external dns_group = external version = 2
19792025-04-30T06:13:08.151ZDEBGomicron-dev (BackgroundTasks): reading DNS config background_task = dns_config_external dns_group = external
19802025-04-30T06:13:08.151ZDEBGomicron-dev (BackgroundTasks): reading DNS zones background_task = dns_config_external dns_group = external
19812025-04-30T06:13:08.151ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
19822025-04-30T06:13:08.151ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
19832025-04-30T06:13:08.151ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
19842025-04-30T06:13:08.151ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
19852025-04-30T06:13:08.151ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
19862025-04-30T06:13:08.152ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
19872025-04-30T06:13:08.153ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b16713c-473c-4a12-9535-4c5c7493012e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19882025-04-30T06:13:08.153ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
19892025-04-30T06:13:08.159ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = DnsConfig result = Ok(())
19902025-04-30T06:13:08.159ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database
19912025-04-30T06:13:08.159ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_internal dns_group = internal roles = RoleSet { roles: {} }
19922025-04-30T06:13:08.160ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_internal dns_group = internal resource = Database result = Ok(())
19932025-04-30T06:13:08.161ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9f54d2e-58a0-4036-886d-a0659384d15e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19942025-04-30T06:13:08.162ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
19952025-04-30T06:13:08.163ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
19962025-04-30T06:13:08.166ZINFOomicron-dev (BackgroundTasks): found Silo/DNS/TLS config (initial) background_task = external_endpoints config = ExternalEndpoints { by_dns_name: {"default-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 001de000-5110-4000-8000-000000000000, db_silo: Silo { identity: SiloIdentity { id: 001de000-5110-4000-8000-000000000000, name: Name(Name("default-silo")), description: "default silo", time_created: 2025-04-30T06:12:58.532310Z, time_modified: 2025-04-30T06:12:58.532310Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {}, rcgen: Generation(Generation(1)) }, tls_certs: [] }, "test-suite-silo.sys.oxide-dev.test": ExternalEndpoint { silo_id: 5f8b1342-cf5b-46a4-a95c-4e40f3bc0974, db_silo: Silo { identity: SiloIdentity { id: 5f8b1342-cf5b-46a4-a95c-4e40f3bc0974, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2025-04-30T06:13:07.360874Z, time_modified: 2025-04-30T06:13:07.360874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }}, warnings: [NoSiloCerts { silo_id: 001de000-5110-4000-8000-000000000000, dns_name: "default-silo.sys.oxide-dev.test" }, NoSiloCerts { silo_id: 5f8b1342-cf5b-46a4-a95c-4e40f3bc0974, dns_name: "test-suite-silo.sys.oxide-dev.test" }], default_endpoint: Some(ExternalEndpoint { silo_id: 5f8b1342-cf5b-46a4-a95c-4e40f3bc0974, db_silo: Silo { identity: SiloIdentity { id: 5f8b1342-cf5b-46a4-a95c-4e40f3bc0974, name: Name(Name("test-suite-silo")), description: "built-in recovery Silo", time_created: 2025-04-30T06:13:07.360874Z, time_modified: 2025-04-30T06:13:07.360874Z, time_deleted: None }, discoverable: false, authentication_mode: Local, user_provision_type: ApiOnly, mapped_fleet_roles: Object {"admin": Array [String("admin")]}, rcgen: Generation(Generation(1)) }, tls_certs: [] }) }
19972025-04-30T06:13:08.166ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = external_endpoints elapsed = 168.426338ms iteration = 2 status = {"by_dns_name":{"default-silo.sys.oxide-dev.test":{"silo_id":"001de000-5110-4000-8000-000000000000","tls_certs":[]},"test-suite-silo.sys.oxide-dev.test":{"silo_id":"5f8b1342-cf5b-46a4-a95c-4e40f3bc0974","tls_certs":[]}},"default_endpoint":{"silo_id":"5f8b1342-cf5b-46a4-a95c-4e40f3bc0974","tls_certs":[]},"warnings":["silo 001de000-5110-4000-8000-000000000000 with DNS name \\"default-silo.sys.oxide-dev.test\\" has no usable certificates","silo 5f8b1342-cf5b-46a4-a95c-4e40f3bc0974 with DNS name \\"test-suite-silo.sys.oxide-dev.test\\" has no usable certificates"]}
19982025-04-30T06:13:08.166ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "14cc7d0c-fa00-48b6-a9d2-841510ee2164", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
19992025-04-30T06:13:08.166ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
20002025-04-30T06:13:08.167ZDEBGomicron-dev (BackgroundTasks): found all DNS names for zone background_task = dns_config_internal dns_group = internal dns_zone_id = e4b87b6b-5de9-4b2c-9d61-c510323fdb21 dns_zone_name = control-plane.oxide.internal found_so_far = 21 version = 1
20012025-04-30T06:13:08.168ZDEBGomicron-dev (BackgroundTasks): read DNS config background_task = dns_config_internal dns_group = internal nzones = 1 version = 1
20022025-04-30T06:13:08.168ZINFOomicron-dev (BackgroundTasks): found latest generation (first find) background_task = dns_config_internal dns_group = internal generation = 1
20032025-04-30T06:13:08.168ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_config_internal dns_group = internal elapsed = 170.589008ms iteration = 2 status = {"generation":1}
20042025-04-30T06:13:08.168ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_propagation_internal dns_group = internal iteration = 3 reason = Dependency
20052025-04-30T06:13:08.169ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e46885b-ef9c-41e6-b9a4-6daee6d715fb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20062025-04-30T06:13:08.169ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
20072025-04-30T06:13:08.171ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e609a58-5e87-4db1-9490-96aefb8f00c6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20082025-04-30T06:13:08.171ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
20092025-04-30T06:13:08.173ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b69baf31-8020-4026-950e-2724c521a05d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20102025-04-30T06:13:08.173ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
20112025-04-30T06:13:08.174ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2dc65638-ce34-4a4f-8ae9-26e6375db495", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20122025-04-30T06:13:08.174ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
20132025-04-30T06:13:08.176ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "06eb6ec4-4164-4d83-be1b-620bb5f2b81d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20142025-04-30T06:13:08.177ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
20152025-04-30T06:13:08.179ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a4e0636-1b20-44cd-9ac5-d1062faae001", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20162025-04-30T06:13:08.179ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
20172025-04-30T06:13:08.181ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "50aae5f7-3ff0-481b-8c44-d72badc4bd92", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20182025-04-30T06:13:08.181ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
20192025-04-30T06:13:08.181ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
20202025-04-30T06:13:08.181ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20212025-04-30T06:13:08.181ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
20222025-04-30T06:13:08.182ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20232025-04-30T06:13:08.183ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2060685f-f95d-44db-88c7-5aabeb48558b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20242025-04-30T06:13:08.183ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
20252025-04-30T06:13:08.185ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0373ad5-d9ac-4f9d-95a6-9d6b509d53a1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20262025-04-30T06:13:08.185ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
20272025-04-30T06:13:08.187ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d216c07d-e270-4ef2-ab71-1b87737a297d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20282025-04-30T06:13:08.187ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
20292025-04-30T06:13:08.187ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
20302025-04-30T06:13:08.188ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0da76c0f-897c-4407-a543-e3f1e66e10eb", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20312025-04-30T06:13:08.188ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
20322025-04-30T06:13:08.190ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8dba14f0-0fa4-4e9d-b937-e6fe474f3c9c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20332025-04-30T06:13:08.191ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
20342025-04-30T06:13:08.195ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f2a19cd-7993-4616-9bcb-a40e174dab0d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20352025-04-30T06:13:08.195ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
20362025-04-30T06:13:08.195ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "954d89f8-b4ad-4ffe-a2cd-b21de2feaaa8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20372025-04-30T06:13:08.195ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
20382025-04-30T06:13:08.196ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a922bc50-7674-4a65-9b64-451e803667a6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20392025-04-30T06:13:08.196ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
20402025-04-30T06:13:08.198ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdb933a5-bb51-40f3-ab20-f396292367a7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20412025-04-30T06:13:08.198ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
20422025-04-30T06:13:08.199ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5676415d-8260-4e2c-9f9a-81df4276ddff", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20432025-04-30T06:13:08.200ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
20442025-04-30T06:13:08.201ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb39dbc5-fb79-4824-b4ca-f5e2138d883b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20452025-04-30T06:13:08.201ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
20462025-04-30T06:13:08.205ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92a34d56-a23c-4f80-89fe-20fe8e548851", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20472025-04-30T06:13:08.206ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
20482025-04-30T06:13:08.209ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b59dcaf-7620-4ef2-a378-110866728424", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20492025-04-30T06:13:08.211ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
20502025-04-30T06:13:08.213ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "21f9f67a-38c4-4993-be60-89e4fb0441c5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20512025-04-30T06:13:08.213ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
20522025-04-30T06:13:08.216ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ad9d2d9-6c9f-4158-a8dc-528adf344a5f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20532025-04-30T06:13:08.217ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
20542025-04-30T06:13:08.223ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e83c2d0f-91ed-4848-9c01-b41608327ee5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20552025-04-30T06:13:08.223ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
20562025-04-30T06:13:08.224ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
20572025-04-30T06:13:08.224ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
20582025-04-30T06:13:08.224ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
20592025-04-30T06:13:08.226ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
20602025-04-30T06:13:08.238ZDEBGomicron-dev (BackgroundTasks): client request background_task = dns_propagation_internal body = Some(Body) dns_group = internal dns_server_url = http://[::1]:39475 generation = 1 method = PUT servers = DnsServersList { addresses: [[::1]:39475] } uri = http://[::1]:39475/config
20612025-04-30T06:13:08.246ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20622025-04-30T06:13:08.246ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
20632025-04-30T06:13:08.248ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20642025-04-30T06:13:08.248ZINFOomicron-dev (http): accepted connection kind = dns local_addr = [::1]:39475 remote_addr = [::1]:37024
20652025-04-30T06:13:08.251ZTRAComicron-dev (http): incoming request kind = dns local_addr = [::1]:39475 method = PUT remote_addr = [::1]:37024 req_id = 0ddfccb8-868e-4c1a-9e01-2f883b0eb034 uri = /config
20662025-04-30T06:13:08.252ZINFOomicron-dev (store): attempting generation update new_generation = 1 req_id = 0ddfccb8-868e-4c1a-9e01-2f883b0eb034
20672025-04-30T06:13:08.252ZINFOomicron-dev (store): updated generation new_generation = 1 req_id = 0ddfccb8-868e-4c1a-9e01-2f883b0eb034
20682025-04-30T06:13:08.253ZINFOomicron-dev (http): request completed kind = dns latency_us = 2057 local_addr = [::1]:39475 method = PUT remote_addr = [::1]:37024 req_id = 0ddfccb8-868e-4c1a-9e01-2f883b0eb034 response_code = 204 uri = /config
20692025-04-30T06:13:08.257ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20702025-04-30T06:13:08.257ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
20712025-04-30T06:13:08.259ZDEBGomicron-dev (BackgroundTasks): client response background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:39475 generation = 1 result = Ok(Response { url: "http://[::1]:39475/config", status: 204, headers: {"x-request-id": "0ddfccb8-868e-4c1a-9e01-2f883b0eb034", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} }) servers = DnsServersList { addresses: [[::1]:39475] }
20722025-04-30T06:13:08.259ZINFOomicron-dev (BackgroundTasks): DNS server now at generation 1 background_task = dns_propagation_internal dns_group = internal dns_server_url = http://[::1]:39475 generation = 1 servers = DnsServersList { addresses: [[::1]:39475] }
20732025-04-30T06:13:08.260ZINFOomicron-dev (BackgroundTasks): DNS propagation: done background_task = dns_propagation_internal dns_group = internal generation = 1 servers = DnsServersList { addresses: [[::1]:39475] }
20742025-04-30T06:13:08.261ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_propagation_internal dns_group = internal elapsed = 92.951671ms iteration = 3 status = {"generation":1,"server_results":{"[::1]:39475":{"Ok":null}}}
20752025-04-30T06:13:08.262ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20762025-04-30T06:13:08.264ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "815a973f-d6c7-4515-899f-a83727d3454c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20772025-04-30T06:13:08.264ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
20782025-04-30T06:13:08.268ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "241fa07d-d828-4bde-a9fb-f9e70adfa0cc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20792025-04-30T06:13:08.270ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
20802025-04-30T06:13:08.270ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20812025-04-30T06:13:08.270ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
20822025-04-30T06:13:08.270ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db35a49c-61ce-429f-aec3-eebf01687d07", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20832025-04-30T06:13:08.270ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
20842025-04-30T06:13:08.272ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20852025-04-30T06:13:08.274ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c951bce6-198e-4970-87b6-3bde42a5c2a5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20862025-04-30T06:13:08.275ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
20872025-04-30T06:13:08.277ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef161835-aa29-4cf8-abad-7cfefa1f7c1b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:08 GMT"} })
20882025-04-30T06:13:08.277ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
20892025-04-30T06:13:08.280ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) }
20902025-04-30T06:13:08.281ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
20912025-04-30T06:13:08.281ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20922025-04-30T06:13:08.281ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
20932025-04-30T06:13:08.285ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
20942025-04-30T06:13:08.310ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
20952025-04-30T06:13:08.310ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
20962025-04-30T06:13:08.310ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
20972025-04-30T06:13:08.311ZTRAComicron-dev (dns): buffer buffer = Length: 79 (0x4f) bytes\n0000: 04 87 01 00 00 01 00 00 00 00 00 01 10 5f 63 72 ............._cr\n0010: 75 63 69 62 6c 65 2d 70 61 6e 74 72 79 04 5f 74 ucible-pantry._t\n0020: 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 cp.control-plane\n0030: 05 6f 78 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 .oxide.internal.\n0040: 00 21 00 01 00 00 29 04 d0 00 00 00 00 00 00 .!....)........ kind = dns peer_addr = [::1]:52296 req_id = 3d6eed9b-d9dc-4939-b440-32b9aa1f03e5
20982025-04-30T06:13:08.311ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 1159,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 16,\n 95,\n 99,\n 114,\n 117,\n 99,\n 105,\n 98,\n 108,\n 101,\n 45,\n 112,\n 97,\n 110,\n 116,\n 114,\n 121,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52296 req_id = 3d6eed9b-d9dc-4939-b440-32b9aa1f03e5
20992025-04-30T06:13:08.312ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21002025-04-30T06:13:08.312ZDEBGomicron-dev (store): query key key = _crucible-pantry._tcp
21012025-04-30T06:13:08.312ZDEBGomicron-dev (store): query key key = 8353cc71-8d6e-45ed-b15b-a3105d4a652c.host
21022025-04-30T06:13:08.312ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("8353cc71-8d6e-45ed-b15b-a3105d4a652c.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52296 query = LowerQuery { name: LowerName(Name("_crucible-pantry._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_crucible-pantry._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 55020, target: Name("8353cc71-8d6e-45ed-b15b-a3105d4a652c.host.control-plane.oxide.internal") })) }] req_id = 3d6eed9b-d9dc-4939-b440-32b9aa1f03e5
21032025-04-30T06:13:08.313ZTRAComicron-dev (dns): buffer buffer = Length: 99 (0x63) bytes\n0000: c1 40 01 00 00 01 00 00 00 00 00 01 24 38 33 35 .@..........$835\n0010: 33 63 63 37 31 2d 38 64 36 65 2d 34 35 65 64 2d 3cc71-8d6e-45ed-\n0020: 62 31 35 62 2d 61 33 31 30 35 64 34 61 36 35 32 b15b-a3105d4a652\n0030: 63 04 68 6f 73 74 0d 63 6f 6e 74 72 6f 6c 2d 70 c.host.control-p\n0040: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0050: 6e 61 6c 00 00 1c 00 01 00 00 29 04 d0 00 00 00 nal.......).....\n0060: 00 00 00 ... kind = dns peer_addr = [::1]:50526 req_id = 95c75dc4-2a16-4027-b7ec-36fec25090a5
21042025-04-30T06:13:08.313ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49472,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("8353cc71-8d6e-45ed-b15b-a3105d4a652c.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("8353cc71-8d6e-45ed-b15b-a3105d4a652c.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 36,\n 56,\n 51,\n 53,\n 51,\n 99,\n 99,\n 55,\n 49,\n 45,\n 56,\n 100,\n 54,\n 101,\n 45,\n 52,\n 53,\n 101,\n 100,\n 45,\n 98,\n 49,\n 53,\n 98,\n 45,\n 97,\n 51,\n 49,\n 48,\n 53,\n 100,\n 52,\n 97,\n 54,\n 53,\n 50,\n 99,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50526 req_id = 95c75dc4-2a16-4027-b7ec-36fec25090a5
21052025-04-30T06:13:08.314ZDEBGomicron-dev (store): query key key = 8353cc71-8d6e-45ed-b15b-a3105d4a652c.host
21062025-04-30T06:13:08.314ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50526 query = LowerQuery { name: LowerName(Name("8353cc71-8d6e-45ed-b15b-a3105d4a652c.host.control-plane.oxide.internal.")), original: Query { name: Name("8353cc71-8d6e-45ed-b15b-a3105d4a652c.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("8353cc71-8d6e-45ed-b15b-a3105d4a652c.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 95c75dc4-2a16-4027-b7ec-36fec25090a5
21072025-04-30T06:13:08.314ZDEBGomicron-dev (BackgroundTasks): found zones background_task = dns_config_external count = 1 dns_group = external
21082025-04-30T06:13:08.314ZDEBGomicron-dev (BackgroundTasks): listing DNS names for zone background_task = dns_config_external batch_size = 100 dns_group = external dns_zone_id = 42362f95-6752-48bd-b7ef-6b59218dfa63 dns_zone_name = oxide-dev.test found_so_far = 0 version = 2
21092025-04-30T06:13:08.315ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig
21102025-04-30T06:13:08.315ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = dns_config_external dns_group = external resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21112025-04-30T06:13:08.315ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
21122025-04-30T06:13:08.315ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21132025-04-30T06:13:08.315ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21142025-04-30T06:13:08.315ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21152025-04-30T06:13:08.315ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21162025-04-30T06:13:08.316ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21172025-04-30T06:13:08.316ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21182025-04-30T06:13:08.317ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8287c8d-8602-4c5e-a30f-2154c380436e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:07 GMT"} })
21192025-04-30T06:13:08.318ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
21202025-04-30T06:13:08.318ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21212025-04-30T06:13:08.318ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
21222025-04-30T06:13:08.318ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
21232025-04-30T06:13:08.319ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
21242025-04-30T06:13:08.323ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21252025-04-30T06:13:08.338ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ById(001de000-074c-4000-8000-000000000000) } result = Ok(())
21262025-04-30T06:13:08.338ZTRAComicron-dev (InternalApi): successful lookup of siloed resource "Vpc" using built-in user
21272025-04-30T06:13:08.338ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21282025-04-30T06:13:08.338ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21292025-04-30T06:13:08.340ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21302025-04-30T06:13:08.341ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21312025-04-30T06:13:08.344ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21322025-04-30T06:13:08.344ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21332025-04-30T06:13:08.345ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21342025-04-30T06:13:08.345ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21352025-04-30T06:13:08.348ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21362025-04-30T06:13:08.348ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21372025-04-30T06:13:08.349ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21382025-04-30T06:13:08.351ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21392025-04-30T06:13:08.351ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21402025-04-30T06:13:08.353ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21412025-04-30T06:13:08.360ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
21422025-04-30T06:13:08.360ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
21432025-04-30T06:13:08.360ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21442025-04-30T06:13:08.360ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
21452025-04-30T06:13:08.360ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
21462025-04-30T06:13:08.360ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
21472025-04-30T06:13:08.363ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = DnsConfig result = Ok(())
21482025-04-30T06:13:08.363ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database
21492025-04-30T06:13:08.363ZDEBGomicron-dev (BackgroundTasks): roles background_task = dns_config_external dns_group = external roles = RoleSet { roles: {} }
21502025-04-30T06:13:08.364ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = dns_config_external dns_group = external resource = Database result = Ok(())
21512025-04-30T06:13:08.370ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21522025-04-30T06:13:08.372ZDEBGomicron-dev (BackgroundTasks): found all DNS names for zone background_task = dns_config_external dns_group = external dns_zone_id = 42362f95-6752-48bd-b7ef-6b59218dfa63 dns_zone_name = oxide-dev.test found_so_far = 1 version = 2
21532025-04-30T06:13:08.372ZDEBGomicron-dev (BackgroundTasks): read DNS config background_task = dns_config_external dns_group = external nzones = 1 version = 2
21542025-04-30T06:13:08.372ZINFOomicron-dev (BackgroundTasks): found latest generation (first find) background_task = dns_config_external dns_group = external generation = 2
21552025-04-30T06:13:08.372ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_config_external dns_group = external elapsed = 374.90294ms iteration = 2 status = {"generation":2}
21562025-04-30T06:13:08.372ZDEBGomicron-dev (BackgroundTasks): activating background_task = dns_propagation_external dns_group = external iteration = 3 reason = Dependency
21572025-04-30T06:13:08.393ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
21582025-04-30T06:13:08.393ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21592025-04-30T06:13:08.393ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21602025-04-30T06:13:08.393ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21612025-04-30T06:13:08.394ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21622025-04-30T06:13:08.398ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
21632025-04-30T06:13:08.399ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
21642025-04-30T06:13:08.399ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
21652025-04-30T06:13:08.400ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
21662025-04-30T06:13:08.423ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21672025-04-30T06:13:08.423ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21682025-04-30T06:13:08.423ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21692025-04-30T06:13:08.423ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21702025-04-30T06:13:08.428ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21712025-04-30T06:13:08.428ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21722025-04-30T06:13:08.429ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21732025-04-30T06:13:08.429ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21742025-04-30T06:13:08.431ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21752025-04-30T06:13:08.439ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
21762025-04-30T06:13:08.439ZTRAComicron-dev (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
21772025-04-30T06:13:08.439ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") }
21782025-04-30T06:13:08.439ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
21792025-04-30T06:13:08.439ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21802025-04-30T06:13:08.439ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21812025-04-30T06:13:08.440ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21822025-04-30T06:13:08.450ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
21832025-04-30T06:13:08.450ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21842025-04-30T06:13:08.450ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21852025-04-30T06:13:08.451ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21862025-04-30T06:13:08.454ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
21872025-04-30T06:13:08.454ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21882025-04-30T06:13:08.454ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21892025-04-30T06:13:08.455ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21902025-04-30T06:13:08.457ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
21912025-04-30T06:13:08.477ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000001, lookup_type: ByName("external-dns") } result = Ok(())
21922025-04-30T06:13:08.477ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21932025-04-30T06:13:08.477ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21942025-04-30T06:13:08.477ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21952025-04-30T06:13:08.490ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21962025-04-30T06:13:08.490ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
21972025-04-30T06:13:08.492ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
21982025-04-30T06:13:08.500ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
21992025-04-30T06:13:08.500ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22002025-04-30T06:13:08.504ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22012025-04-30T06:13:08.506ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22022025-04-30T06:13:08.506ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22032025-04-30T06:13:08.507ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22042025-04-30T06:13:08.521ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22052025-04-30T06:13:08.521ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22062025-04-30T06:13:08.523ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22072025-04-30T06:13:08.527ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22082025-04-30T06:13:08.527ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22092025-04-30T06:13:08.527ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22102025-04-30T06:13:08.527ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22112025-04-30T06:13:08.528ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22122025-04-30T06:13:08.533ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22132025-04-30T06:13:08.533ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22142025-04-30T06:13:08.533ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22152025-04-30T06:13:08.533ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22162025-04-30T06:13:08.538ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22172025-04-30T06:13:08.538ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22182025-04-30T06:13:08.538ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22192025-04-30T06:13:08.539ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22202025-04-30T06:13:08.541ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22212025-04-30T06:13:08.547ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:43268 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22222025-04-30T06:13:08.548ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:43268 req_id = 6e0db4b2-53f2-4699-afab-994f1069524a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22232025-04-30T06:13:08.548ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
22242025-04-30T06:13:08.548ZTRAComicron-dev (InternalApi): successful lookup of siloed resource "VpcSubnet" using built-in user
22252025-04-30T06:13:08.548ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") }
22262025-04-30T06:13:08.548ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-4401-4000-8000-000000000000 resource_type = Project
22272025-04-30T06:13:08.548ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22282025-04-30T06:13:08.548ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22292025-04-30T06:13:08.549ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22302025-04-30T06:13:08.549ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 672 local_addr = [::1]:55020 method = GET remote_addr = [::1]:43268 req_id = 6e0db4b2-53f2-4699-afab-994f1069524a response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22312025-04-30T06:13:08.557ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-5110-4000-8000-000000000001 resource_type = Silo
22322025-04-30T06:13:08.557ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22332025-04-30T06:13:08.557ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22342025-04-30T06:13:08.558ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22352025-04-30T06:13:09.115ZDEBGomicron-dev (BackgroundTasks): client request background_task = dns_propagation_external body = Some(Body) dns_group = external dns_server_url = http://[::1]:54696 generation = 2 method = PUT servers = DnsServersList { addresses: [[::1]:54696] } uri = http://[::1]:54696/config
22362025-04-30T06:13:09.256ZINFOomicron-dev (http): accepted connection kind = dns local_addr = [::1]:54696 remote_addr = [::1]:46300
22372025-04-30T06:13:09.257ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:56227 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22382025-04-30T06:13:09.258ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22392025-04-30T06:13:09.259ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22402025-04-30T06:13:09.259ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22412025-04-30T06:13:09.259ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:56227 req_id = 19dae95a-5241-4f60-8a61-e817674c4298 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22422025-04-30T06:13:09.259ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22432025-04-30T06:13:09.259ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 374 local_addr = [::1]:55020 method = GET remote_addr = [::1]:56227 req_id = 19dae95a-5241-4f60-8a61-e817674c4298 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22442025-04-30T06:13:09.260ZTRAComicron-dev (http): incoming request kind = dns local_addr = [::1]:54696 method = PUT remote_addr = [::1]:46300 req_id = d3382cf7-1d52-45f3-9dfb-079226673bbe uri = /config
22452025-04-30T06:13:09.260ZINFOomicron-dev (store): attempting generation update new_generation = 2 req_id = d3382cf7-1d52-45f3-9dfb-079226673bbe
22462025-04-30T06:13:09.260ZINFOomicron-dev (store): pruning trees for generations newer than 0
22472025-04-30T06:13:09.260ZDEBGomicron-dev (store): creating tree tree_name = generation_2_zone_oxide-dev.test
22482025-04-30T06:13:09.260ZDEBGomicron-dev (store): updating current config
22492025-04-30T06:13:09.263ZDEBGomicron-dev (store): flushing default tree
22502025-04-30T06:13:09.263ZINFOomicron-dev (store): pruning trees for generations older than 2 keep = 3
22512025-04-30T06:13:09.263ZINFOomicron-dev (store): updated generation new_generation = 2 req_id = d3382cf7-1d52-45f3-9dfb-079226673bbe
22522025-04-30T06:13:09.263ZINFOomicron-dev (http): request completed kind = dns latency_us = 3276 local_addr = [::1]:54696 method = PUT remote_addr = [::1]:46300 req_id = d3382cf7-1d52-45f3-9dfb-079226673bbe response_code = 204 uri = /config
22532025-04-30T06:13:09.265ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22542025-04-30T06:13:09.265ZDEBGomicron-dev (BackgroundTasks): client response background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:54696 generation = 2 result = Ok(Response { url: "http://[::1]:54696/config", status: 204, headers: {"x-request-id": "d3382cf7-1d52-45f3-9dfb-079226673bbe", "date": "Wed, 30 Apr 2025 06:13:09 GMT"} }) servers = DnsServersList { addresses: [[::1]:54696] }
22552025-04-30T06:13:09.265ZINFOomicron-dev (BackgroundTasks): DNS server now at generation 2 background_task = dns_propagation_external dns_group = external dns_server_url = http://[::1]:54696 generation = 2 servers = DnsServersList { addresses: [[::1]:54696] }
22562025-04-30T06:13:09.273ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = VpcSubnet { parent: Vpc { parent: Project { parent: Silo { parent: Fleet, key: 001de000-5110-4000-8000-000000000001, lookup_type: ById(001de000-5110-4000-8000-000000000001) }, key: 001de000-4401-4000-8000-000000000000, lookup_type: ById(001de000-4401-4000-8000-000000000000) }, key: 001de000-074c-4000-8000-000000000000, lookup_type: ByName("oxide-services") }, key: 001de000-c470-4000-8000-000000000002, lookup_type: ByName("nexus") } result = Ok(())
22572025-04-30T06:13:09.273ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22582025-04-30T06:13:09.273ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22592025-04-30T06:13:09.273ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22602025-04-30T06:13:09.414ZINFOomicron-dev (BackgroundTasks): DNS propagation: done background_task = dns_propagation_external dns_group = external generation = 2 servers = DnsServersList { addresses: [[::1]:54696] }
22612025-04-30T06:13:09.415ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = dns_propagation_external dns_group = external elapsed = 1.042384483s iteration = 3 status = {"generation":2,"server_results":{"[::1]:54696":{"Ok":null}}}
22622025-04-30T06:13:09.460ZDEBGomicron-dev (InternalApi): resolved names for firewall rules instance_interfaces = {} subnet_interfaces = {Name("external-dns"): [NetworkInterface { id: 925b1d1e-94b1-469c-94f6-b225eb8d1c11, kind: Service { id: 90372d2a-e95e-4302-b338-2843dd07d33a }, name: Name("external-dns-90372d2a-e95e-4302-b338-2843dd07d33a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }], Name("nexus"): [NetworkInterface { id: cf66f81f-75b3-463a-96ab-2fe8219455e6, kind: Service { id: e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c }, name: Name("nexus-e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }]} subnet_networks = {Name("external-dns"): [V4(Ipv4Network { addr: 172.30.1.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:1::, prefix: 64 })], Name("nexus"): [V4(Ipv4Network { addr: 172.30.2.0, prefix: 24 }), V6(Ipv6Network { addr: fd77:e9d2:9cd9:2::, prefix: 64 })]} vpc_interfaces = {}
22632025-04-30T06:13:09.460ZTRAComicron-dev (InternalApi): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
22642025-04-30T06:13:09.460ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22652025-04-30T06:13:09.460ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22662025-04-30T06:13:09.460ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22672025-04-30T06:13:09.461ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22682025-04-30T06:13:09.474ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22692025-04-30T06:13:09.478ZDEBGomicron-dev (InternalApi): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
22702025-04-30T06:13:09.478ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22712025-04-30T06:13:09.478ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22722025-04-30T06:13:09.479ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22732025-04-30T06:13:09.482ZTRAComicron-dev (InternalApi): fetched allowlist from DB allowed = AllowList { id: 001de000-a110-4000-8000-000000000000, time_created: 2025-04-30T06:06:08.248500Z, time_modified: 2025-04-30T06:13:07.532584Z, allowed_ips: None }
22742025-04-30T06:13:09.482ZDEBGomicron-dev (InternalApi): Allowlist for user-facing services is set to allow any inbound traffic. Existing VPC firewall rules will not be modified.
22752025-04-30T06:13:09.483ZDEBGomicron-dev (InternalApi): resolved firewall rules for sled agents sled_agent_rules = [ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("53")]), filter_protocols: Some([Udp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: 925b1d1e-94b1-469c-94f6-b225eb8d1c11, kind: Service { id: 90372d2a-e95e-4302-b338-2843dd07d33a }, name: Name("external-dns-90372d2a-e95e-4302-b338-2843dd07d33a"), ip: 172.30.1.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 0])), subnet: V4(Ipv4Net { addr: 172.30.1.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }, ResolvedVpcFirewallRule { action: Allow, direction: Inbound, filter_hosts: None, filter_ports: Some([L4PortRange("80"), L4PortRange("443")]), filter_protocols: Some([Tcp]), priority: 65534, status: Enabled, targets: [NetworkInterface { id: cf66f81f-75b3-463a-96ab-2fe8219455e6, kind: Service { id: e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c }, name: Name("nexus-e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c"), ip: 172.30.2.6, mac: MacAddr(MacAddr6([168, 64, 37, 255, 128, 1])), subnet: V4(Ipv4Net { addr: 172.30.2.0, width: 24 }), vni: Vni(100), primary: true, slot: 0, transit_ips: [] }] }]
22762025-04-30T06:13:09.483ZDEBGomicron-dev (InternalApi): resolved 2 rules for sleds
22772025-04-30T06:13:09.561ZDEBGomicron-dev (InternalApi): resolved sleds for vpc oxide-services vpc_to_sled = [Sled { identity: SledIdentity { id: b6d65341-167c-41df-9b5c-41cded99c229, time_created: 2025-04-30T06:13:01.188956Z, time_modified: 2025-04-30T06:13:01.188956Z }, time_deleted: None, rcgen: Generation(Generation(1)), rack_id: c19a698f-c6f9-4a17-ae30-20d711b8f7dc, is_scrimlet: true, serial_number: "sim-b6d65341", part_number: "Unknown", revision: SqlU32(0), usable_hardware_threads: SqlU32(16), usable_physical_ram: ByteCount(ByteCount(34359738368)), reservoir_size: ByteCount(ByteCount(17179869184)), ip: ::1, port: SqlU16(61881), last_used_address: ::ffff, policy: InService, state: Active, sled_agent_gen: Generation(Generation(1)), repo_depot_port: SqlU16(43978) }]
22782025-04-30T06:13:09.561ZDEBGomicron-dev (InternalApi): sending firewall rules to sled agents
22792025-04-30T06:13:09.561ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22802025-04-30T06:13:09.561ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22812025-04-30T06:13:09.565ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22822025-04-30T06:13:09.568ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
22832025-04-30T06:13:09.568ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
22842025-04-30T06:13:09.568ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
22852025-04-30T06:13:09.568ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
22862025-04-30T06:13:09.569ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
22872025-04-30T06:13:09.571ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
22882025-04-30T06:13:09.576ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
22892025-04-30T06:13:09.665ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
22902025-04-30T06:13:09.665ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
22912025-04-30T06:13:09.666ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:59558 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22922025-04-30T06:13:09.667ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:59558 req_id = d39bb626-d6ce-4b23-9946-3a85a93effc4 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
22932025-04-30T06:13:09.668ZINFOomicron-dev (dropshot): request completed latency_us = 719 local_addr = [::1]:61881 method = GET remote_addr = [::1]:59558 req_id = d39bb626-d6ce-4b23-9946-3a85a93effc4 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
22942025-04-30T06:13:09.668ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d39bb626-d6ce-4b23-9946-3a85a93effc4", "content-length": "2756", "date": "Wed, 30 Apr 2025 06:13:09 GMT"} })
22952025-04-30T06:13:09.838ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:42978 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
22962025-04-30T06:13:09.839ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:42978 req_id = edc6d0cd-265f-4f1e-a158-0ff19f501a14 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22972025-04-30T06:13:09.839ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 291 local_addr = [::1]:55020 method = GET remote_addr = [::1]:42978 req_id = edc6d0cd-265f-4f1e-a158-0ff19f501a14 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
22982025-04-30T06:13:10.122ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
22992025-04-30T06:13:10.123ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
23002025-04-30T06:13:10.123ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
23012025-04-30T06:13:10.180ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:44157 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
23022025-04-30T06:13:10.181ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:44157 req_id = 95f512b3-d4c4-4346-a675-e6f9bc4cda69 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23032025-04-30T06:13:10.181ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 294 local_addr = [::1]:55020 method = GET remote_addr = [::1]:44157 req_id = 95f512b3-d4c4-4346-a675-e6f9bc4cda69 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23042025-04-30T06:13:10.220ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
23052025-04-30T06:13:10.221ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23062025-04-30T06:13:10.221ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
23072025-04-30T06:13:10.221ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
23082025-04-30T06:13:10.222ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
23092025-04-30T06:13:10.224ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:49118 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
23102025-04-30T06:13:10.225ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:49118 req_id = 770f7da7-5d96-43c3-9635-72a3542e11ab sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23112025-04-30T06:13:10.225ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 262 local_addr = [::1]:55020 method = GET remote_addr = [::1]:49118 req_id = 770f7da7-5d96-43c3-9635-72a3542e11ab response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23122025-04-30T06:13:10.226ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23132025-04-30T06:13:10.229ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
23142025-04-30T06:13:10.230ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
23152025-04-30T06:13:10.230ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
23162025-04-30T06:13:10.231ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
23172025-04-30T06:13:10.243ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:51622 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
23182025-04-30T06:13:10.244ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:51622 req_id = 2273234a-48c0-43b0-ba1b-5e9c5570ab9b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23192025-04-30T06:13:10.244ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 264 local_addr = [::1]:55020 method = GET remote_addr = [::1]:51622 req_id = 2273234a-48c0-43b0-ba1b-5e9c5570ab9b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23202025-04-30T06:13:10.281ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:47559 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
23212025-04-30T06:13:10.283ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:47559 req_id = 98222664-6f64-4a2f-9fb6-5eb454fb09a3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23222025-04-30T06:13:10.283ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 323 local_addr = [::1]:55020 method = GET remote_addr = [::1]:47559 req_id = 98222664-6f64-4a2f-9fb6-5eb454fb09a3 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23232025-04-30T06:13:10.316ZDEBGomicron-dev (InternalApi): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 body = Some(Body) method = PUT uri = http://[::1]:61881/vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23242025-04-30T06:13:10.317ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:51732 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
23252025-04-30T06:13:10.318ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = PUT remote_addr = [::1]:51732 req_id = c101f25c-2d53-407b-9e3d-3207251fa794 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23262025-04-30T06:13:10.320ZINFOomicron-dev (dropshot): request completed latency_us = 1613 local_addr = [::1]:61881 method = PUT remote_addr = [::1]:51732 req_id = c101f25c-2d53-407b-9e3d-3207251fa794 response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc/001de000-074c-4000-8000-000000000000/firewall/rules
23272025-04-30T06:13:10.322ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:53313 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
23282025-04-30T06:13:10.324ZDEBGomicron-dev (InternalApi): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 result = Ok(Response { url: "http://[::1]:61881/vpc/001de000-074c-4000-8000-000000000000/firewall/rules", status: 204, headers: {"x-request-id": "c101f25c-2d53-407b-9e3d-3207251fa794", "date": "Wed, 30 Apr 2025 06:13:10 GMT"} })
23292025-04-30T06:13:10.324ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:53313 req_id = d3ed335f-9500-4ced-a926-cdda1b4590f5 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23302025-04-30T06:13:10.325ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 527 local_addr = [::1]:55020 method = GET remote_addr = [::1]:53313 req_id = d3ed335f-9500-4ced-a926-cdda1b4590f5 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
23312025-04-30T06:13:10.325ZINFOomicron-dev (InternalApi): updated firewall rules on 1 sleds
23322025-04-30T06:13:10.325ZINFOomicron-dev (nexus): plumbed initial IP allowlist
23332025-04-30T06:13:10.328ZDEBGomicron-dev (BackgroundTasks): activating background_task = blueprint_loader iteration = 2 reason = Timeout
23342025-04-30T06:13:10.328ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
23352025-04-30T06:13:10.328ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23362025-04-30T06:13:10.328ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
23372025-04-30T06:13:10.328ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23382025-04-30T06:13:10.330ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23392025-04-30T06:13:10.334ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23402025-04-30T06:13:10.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
23412025-04-30T06:13:10.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
23422025-04-30T06:13:10.339ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23432025-04-30T06:13:10.340ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23442025-04-30T06:13:10.356ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bfc4639-5ca6-4323-a678-d4a0347a6fdf, lookup_type: ById(1bfc4639-5ca6-4323-a678-d4a0347a6fdf) }
23452025-04-30T06:13:10.356ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23462025-04-30T06:13:10.356ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
23472025-04-30T06:13:10.356ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23482025-04-30T06:13:10.360ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23492025-04-30T06:13:10.363ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23502025-04-30T06:13:10.373ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bfc4639-5ca6-4323-a678-d4a0347a6fdf, lookup_type: ById(1bfc4639-5ca6-4323-a678-d4a0347a6fdf) } result = Ok(())
23512025-04-30T06:13:10.373ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
23522025-04-30T06:13:10.373ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
23532025-04-30T06:13:10.373ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
23542025-04-30T06:13:10.428ZINFOomicron-dev (BackgroundTasks): found new target blueprint (first find) background_task = blueprint_loader target_id = 1bfc4639-5ca6-4323-a678-d4a0347a6fdf time_created = 2025-04-30 06:13:06.783824 UTC
23552025-04-30T06:13:10.434ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 105.9811ms iteration = 2 status = {"enabled":false,"status":"first target blueprint","target_id":"1bfc4639-5ca6-4323-a678-d4a0347a6fdf","time_created":"2025-04-30T06:13:06.783824Z","time_found":"2025-04-30T06:13:10.434081786Z"}
23562025-04-30T06:13:10.434ZDEBGomicron-dev (BackgroundTasks): activating background_task = crdb_node_id_collector iteration = 2 reason = Dependency
23572025-04-30T06:13:10.434ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet
23582025-04-30T06:13:10.434ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = crdb_node_id_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
23592025-04-30T06:13:10.435ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database
23602025-04-30T06:13:10.435ZDEBGomicron-dev (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
23612025-04-30T06:13:10.435ZDEBGomicron-dev (BackgroundTasks): activating background_task = blueprint_executor iteration = 2 reason = Dependency
23622025-04-30T06:13:10.435ZWARNomicron-dev (BackgroundTasks): Blueprint execution: skipped background_task = blueprint_executor reason = blueprint disabled target_id = 1bfc4639-5ca6-4323-a678-d4a0347a6fdf
23632025-04-30T06:13:10.435ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = blueprint_executor elapsed = 65.521\u{b5}s iteration = 2 status = {"enabled":false,"target_id":"1bfc4639-5ca6-4323-a678-d4a0347a6fdf"}
23642025-04-30T06:13:10.435ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
23652025-04-30T06:13:10.438ZDEBGomicron-dev (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
23662025-04-30T06:13:10.442ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Fleet result = Ok(())
23672025-04-30T06:13:10.442ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database
23682025-04-30T06:13:10.442ZDEBGomicron-dev (BackgroundTasks): roles background_task = crdb_node_id_collector roles = RoleSet { roles: {} }
23692025-04-30T06:13:10.442ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = crdb_node_id_collector resource = Database result = Ok(())
23702025-04-30T06:13:10.499ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = / versions = all
23712025-04-30T06:13:10.505ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /access versions = all
23722025-04-30T06:13:10.505ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /assets/{path} versions = all
23732025-04-30T06:13:10.505ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /device/auth versions = all
23742025-04-30T06:13:10.505ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /device/confirm versions = all
23752025-04-30T06:13:10.505ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /device/success versions = all
23762025-04-30T06:13:10.505ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /device/token versions = all
23772025-04-30T06:13:10.506ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /device/verify versions = all
23782025-04-30T06:13:10.506ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /experimental/v1/probes versions = all
23792025-04-30T06:13:10.506ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /experimental/v1/probes versions = all
23802025-04-30T06:13:10.506ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /experimental/v1/probes/{probe} versions = all
23812025-04-30T06:13:10.506ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /experimental/v1/probes/{probe} versions = all
23822025-04-30T06:13:10.506ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /experimental/v1/system/support-bundles versions = all
23832025-04-30T06:13:10.506ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /experimental/v1/system/support-bundles versions = all
23842025-04-30T06:13:10.506ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
23852025-04-30T06:13:10.506ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
23862025-04-30T06:13:10.507ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
23872025-04-30T06:13:10.507ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
23882025-04-30T06:13:10.507ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
23892025-04-30T06:13:10.507ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
23902025-04-30T06:13:10.507ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
23912025-04-30T06:13:10.507ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /images versions = all
23922025-04-30T06:13:10.508ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /login versions = all
23932025-04-30T06:13:10.508ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /login/{silo_name}/local versions = all
23942025-04-30T06:13:10.508ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
23952025-04-30T06:13:10.508ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
23962025-04-30T06:13:10.508ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
23972025-04-30T06:13:10.508ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /lookup/{path} versions = all
23982025-04-30T06:13:10.508ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /projects/{path} versions = all
23992025-04-30T06:13:10.508ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /projects-new versions = all
24002025-04-30T06:13:10.509ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /settings/{path} versions = all
24012025-04-30T06:13:10.509ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /system/{path} versions = all
24022025-04-30T06:13:10.509ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /utilization versions = all
24032025-04-30T06:13:10.509ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/affinity-groups versions = all
24042025-04-30T06:13:10.509ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/affinity-groups versions = all
24052025-04-30T06:13:10.509ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
24062025-04-30T06:13:10.509ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
24072025-04-30T06:13:10.509ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
24082025-04-30T06:13:10.510ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
24092025-04-30T06:13:10.510ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24102025-04-30T06:13:10.510ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24112025-04-30T06:13:10.510ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
24122025-04-30T06:13:10.510ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/anti-affinity-groups versions = all
24132025-04-30T06:13:10.510ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/anti-affinity-groups versions = all
24142025-04-30T06:13:10.511ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24152025-04-30T06:13:10.511ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24162025-04-30T06:13:10.511ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
24172025-04-30T06:13:10.511ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
24182025-04-30T06:13:10.511ZDEBGomicron-dev (BackgroundTasks): client request background_task = crdb_node_id_collector body = None method = GET uri = http://[::1]:32222/node/id
24192025-04-30T06:13:10.511ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24202025-04-30T06:13:10.511ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24212025-04-30T06:13:10.511ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
24222025-04-30T06:13:10.511ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/certificates versions = all
24232025-04-30T06:13:10.512ZDEBGomicron-dev (BackgroundTasks): client response background_task = crdb_node_id_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:32222/node/id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
24242025-04-30T06:13:10.512ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/certificates versions = all
24252025-04-30T06:13:10.512ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/certificates/{certificate} versions = all
24262025-04-30T06:13:10.512ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/certificates/{certificate} versions = all
24272025-04-30T06:13:10.512ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/disks versions = all
24282025-04-30T06:13:10.512ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/disks versions = all
24292025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/disks/{disk} versions = all
24302025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/disks/{disk} versions = all
24312025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/disks/{disk}/bulk-write versions = all
24322025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
24332025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
24342025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/disks/{disk}/finalize versions = all
24352025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
24362025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/floating-ips versions = all
24372025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/floating-ips versions = all
24382025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
24392025-04-30T06:13:10.513ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/floating-ips/{floating_ip} versions = all
24402025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
24412025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
24422025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
24432025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/groups versions = all
24442025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/groups/{group_id} versions = all
24452025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/images versions = all
24462025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/images versions = all
24472025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/images/{image} versions = all
24482025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/images/{image} versions = all
24492025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/images/{image}/demote versions = all
24502025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/images/{image}/promote versions = all
24512025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/instances versions = all
24522025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/instances versions = all
24532025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/instances/{instance} versions = all
24542025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/instances/{instance} versions = all
24552025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/instances/{instance} versions = all
24562025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
24572025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
24582025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/instances/{instance}/disks versions = all
24592025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/instances/{instance}/disks/attach versions = all
24602025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/instances/{instance}/disks/detach versions = all
24612025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/instances/{instance}/external-ips versions = all
24622025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24632025-04-30T06:13:10.514ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
24642025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/instances/{instance}/reboot versions = all
24652025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/instances/{instance}/serial-console versions = all
24662025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
24672025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
24682025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/instances/{instance}/start versions = all
24692025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/instances/{instance}/stop versions = all
24702025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/internet-gateway-ip-addresses versions = all
24712025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/internet-gateway-ip-addresses versions = all
24722025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
24732025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/internet-gateway-ip-pools versions = all
24742025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/internet-gateway-ip-pools versions = all
24752025-04-30T06:13:10.515ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
24762025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/internet-gateways versions = all
24772025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/internet-gateways versions = all
24782025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
24792025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/internet-gateways/{gateway} versions = all
24802025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/ip-pools versions = all
24812025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/ip-pools/{pool} versions = all
24822025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/login/{silo_name}/local versions = all
24832025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/logout versions = all
24842025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/me versions = all
24852025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/me/groups versions = all
24862025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/me/ssh-keys versions = all
24872025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/me/ssh-keys versions = all
24882025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
24892025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
24902025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/metrics/{metric_name} versions = all
24912025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/network-interfaces versions = all
24922025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/network-interfaces versions = all
24932025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/network-interfaces/{interface} versions = all
24942025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/network-interfaces/{interface} versions = all
24952025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/network-interfaces/{interface} versions = all
24962025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/ping versions = all
24972025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/policy versions = all
24982025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/policy versions = all
24992025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/projects versions = all
25002025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/projects versions = all
25012025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/projects/{project} versions = all
25022025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/projects/{project} versions = all
25032025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/projects/{project} versions = all
25042025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/projects/{project}/policy versions = all
25052025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/projects/{project}/policy versions = all
25062025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/snapshots versions = all
25072025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/snapshots versions = all
25082025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/snapshots/{snapshot} versions = all
25092025-04-30T06:13:10.516ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/snapshots/{snapshot} versions = all
25102025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/disks versions = all
25112025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
25122025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
25132025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/racks versions = all
25142025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
25152025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/sleds versions = all
25162025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/hardware/sleds versions = all
25172025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
25182025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
25192025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
25202025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
25212025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
25222025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/switch-port versions = all
25232025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
25242025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
25252025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
25262025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
25272025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
25282025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/switches versions = all
25292025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
25302025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/identity-providers versions = all
25312025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/identity-providers/local/users versions = all
25322025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
25332025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
25342025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/identity-providers/saml versions = all
25352025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
25362025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/ip-pools versions = all
25372025-04-30T06:13:10.517ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/ip-pools versions = all
25382025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
25392025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/ip-pools/{pool} versions = all
25402025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/ip-pools/{pool} versions = all
25412025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
25422025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
25432025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
25442025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
25452025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
25462025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25472025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
25482025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
25492025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/ip-pools-service versions = all
25502025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/ip-pools-service/ranges versions = all
25512025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
25522025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
25532025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/metrics/{metric_name} versions = all
25542025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/address-lot versions = all
25552025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/networking/address-lot versions = all
25562025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
25572025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
25582025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/allow-list versions = all
25592025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/networking/allow-list versions = all
25602025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/networking/bfd-disable versions = all
25612025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/networking/bfd-enable versions = all
25622025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/bfd-status versions = all
25632025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/networking/bgp versions = all
25642025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/bgp versions = all
25652025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/networking/bgp versions = all
25662025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/bgp-announce-set versions = all
25672025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
25682025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
25692025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
25702025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/bgp-exported versions = all
25712025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/bgp-message-history versions = all
25722025-04-30T06:13:10.518ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
25732025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/bgp-status versions = all
25742025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/loopback-address versions = all
25752025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/networking/loopback-address versions = all
25762025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
25772025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
25782025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/switch-port-settings versions = all
25792025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/networking/switch-port-settings versions = all
25802025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
25812025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/policy versions = all
25822025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/policy versions = all
25832025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/roles versions = all
25842025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/roles/{role_name} versions = all
25852025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/silo-quotas versions = all
25862025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/silos versions = all
25872025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/silos versions = all
25882025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/system/silos/{silo} versions = all
25892025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/silos/{silo} versions = all
25902025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
25912025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/silos/{silo}/policy versions = all
25922025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/silos/{silo}/policy versions = all
25932025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/silos/{silo}/quotas versions = all
25942025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
25952025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/system/timeseries/query versions = all
25962025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/timeseries/schemas versions = all
25972025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/update/repository versions = all
25982025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/update/repository/{system_version} versions = all
25992025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/update/target-release versions = all
26002025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/system/update/target-release versions = all
26012025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/users versions = all
26022025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/users/{user_id} versions = all
26032025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/users-builtin versions = all
26042025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/users-builtin/{user} versions = all
26052025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/utilization/silos versions = all
26062025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/system/utilization/silos/{silo} versions = all
26072025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/timeseries/query versions = all
26082025-04-30T06:13:10.519ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/users versions = all
26092025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/utilization versions = all
26102025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpc-firewall-rules versions = all
26112025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/vpc-firewall-rules versions = all
26122025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpc-router-routes versions = all
26132025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/vpc-router-routes versions = all
26142025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
26152025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpc-router-routes/{route} versions = all
26162025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/vpc-router-routes/{route} versions = all
26172025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpc-routers versions = all
26182025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/vpc-routers versions = all
26192025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/vpc-routers/{router} versions = all
26202025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpc-routers/{router} versions = all
26212025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/vpc-routers/{router} versions = all
26222025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpc-subnets versions = all
26232025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/vpc-subnets versions = all
26242025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
26252025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpc-subnets/{subnet} versions = all
26262025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
26272025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
26282025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpcs versions = all
26292025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/vpcs versions = all
26302025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/vpcs/{vpc} versions = all
26312025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/vpcs/{vpc} versions = all
26322025-04-30T06:13:10.520ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = crdb_node_id_collector elapsed = 82.653772ms iteration = 2 status = {"errors":[{"err":"failed to fetch node ID for zone bb6e149b-0d6f-4f9e-8216-e4a59d897bce at http://[::1]:32222: Communication Error: error sending request for url (http://[::1]:32222/node/id): error sending request for url (http://[::1]:32222/node/id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146)","zone_id":"bb6e149b-0d6f-4f9e-8216-e4a59d897bce"}],"nsuccess":0}
26332025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/vpcs/{vpc} versions = all
26342025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/webhooks/deliveries versions = all
26352025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/webhooks/deliveries/{event_id}/resend versions = all
26362025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/webhooks/event-classes versions = all
26372025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/webhooks/receivers versions = all
26382025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/webhooks/receivers versions = all
26392025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/webhooks/receivers/{receiver} versions = all
26402025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/webhooks/receivers/{receiver} versions = all
26412025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = PUT path = /v1/webhooks/receivers/{receiver} versions = all
26422025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/webhooks/receivers/{receiver}/probe versions = all
26432025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/webhooks/receivers/{receiver}/subscriptions versions = all
26442025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/webhooks/receivers/{receiver}/subscriptions/{subscription} versions = all
26452025-04-30T06:13:10.520ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = GET path = /v1/webhooks/secrets versions = all
26462025-04-30T06:13:10.521ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = POST path = /v1/webhooks/secrets versions = all
26472025-04-30T06:13:10.521ZDEBGomicron-dev (dropshot_external): registered endpoint local_addr = 127.0.0.1:63407 method = DELETE path = /v1/webhooks/secrets/{secret_id} versions = all
26482025-04-30T06:13:10.521ZINFOomicron-dev (dropshot_external): listening local_addr = 127.0.0.1:63407
26492025-04-30T06:13:10.521ZDEBGomicron-dev (dropshot_external): successfully registered DTrace USDT probes local_addr = 127.0.0.1:63407
26502025-04-30T06:13:10.592ZINFOomicron-dev (BackgroundTasks): inserted inventory collection background_task = inventory_collection collection_id = 56c2f67f-67f1-43f2-8f3d-f724f719303e
26512025-04-30T06:13:10.593ZDEBGomicron-dev (BackgroundTasks): inventory collection complete background_task = inventory_collection collection_id = 56c2f67f-67f1-43f2-8f3d-f724f719303e time_started = 2025-04-30 06:13:08.066222 UTC
26522025-04-30T06:13:10.593ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.595100097s iteration = 3 status = {"collection_id":"56c2f67f-67f1-43f2-8f3d-f724f719303e","time_done":"2025-04-30 06:13:10.122992 UTC","time_started":"2025-04-30 06:13:08.066222 UTC"}
26532025-04-30T06:13:10.593ZDEBGomicron-dev (BackgroundTasks): activating background_task = blueprint_rendezvous iteration = 2 reason = Dependency
26542025-04-30T06:13:10.593ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory
26552025-04-30T06:13:10.593ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_rendezvous resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
26562025-04-30T06:13:10.593ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database
26572025-04-30T06:13:10.593ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
26582025-04-30T06:13:10.593ZDEBGomicron-dev (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 2 reason = Dependency
26592025-04-30T06:13:10.594ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet
26602025-04-30T06:13:10.594ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = physical_disk_adoption resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
26612025-04-30T06:13:10.594ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
26622025-04-30T06:13:10.594ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26632025-04-30T06:13:10.594ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
26642025-04-30T06:13:10.595ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26652025-04-30T06:13:10.601ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26662025-04-30T06:13:10.602ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26672025-04-30T06:13:10.606ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26682025-04-30T06:13:10.606ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
26692025-04-30T06:13:10.606ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26702025-04-30T06:13:10.607ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Inventory result = Ok(())
26712025-04-30T06:13:10.607ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database
26722025-04-30T06:13:10.607ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
26732025-04-30T06:13:10.607ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26742025-04-30T06:13:10.607ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
26752025-04-30T06:13:10.612ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database
26762025-04-30T06:13:10.612ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
26772025-04-30T06:13:10.613ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet
26782025-04-30T06:13:10.613ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = physical_disk_adoption resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
26792025-04-30T06:13:10.613ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
26802025-04-30T06:13:10.613ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26812025-04-30T06:13:10.613ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
26822025-04-30T06:13:10.613ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26832025-04-30T06:13:10.619ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
26842025-04-30T06:13:10.624ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
26852025-04-30T06:13:10.624ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
26862025-04-30T06:13:10.624ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
26872025-04-30T06:13:10.624ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
26882025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = / versions = all
26892025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /access versions = all
26902025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /assets/{path} versions = all
26912025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /device/auth versions = all
26922025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /device/confirm versions = all
26932025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /device/success versions = all
26942025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /device/token versions = all
26952025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /device/verify versions = all
26962025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /experimental/v1/probes versions = all
26972025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /experimental/v1/probes versions = all
26982025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /experimental/v1/probes/{probe} versions = all
26992025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /experimental/v1/probes/{probe} versions = all
27002025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /experimental/v1/system/support-bundles versions = all
27012025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /experimental/v1/system/support-bundles versions = all
27022025-04-30T06:13:10.665ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
27032025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /experimental/v1/system/support-bundles/{support_bundle} versions = all
27042025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
27052025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download versions = all
27062025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
27072025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = HEAD path = /experimental/v1/system/support-bundles/{support_bundle}/download/{file} versions = all
27082025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /experimental/v1/system/support-bundles/{support_bundle}/index versions = all
27092025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /images versions = all
27102025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /login versions = all
27112025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /login/{silo_name}/local versions = all
27122025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /login/{silo_name}/saml/{provider_name} versions = all
27132025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /login/{silo_name}/saml/{provider_name} versions = all
27142025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /login/{silo_name}/saml/{provider_name}/redirect versions = all
27152025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /lookup/{path} versions = all
27162025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /projects/{path} versions = all
27172025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /projects-new versions = all
27182025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /settings/{path} versions = all
27192025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /system/{path} versions = all
27202025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /utilization versions = all
27212025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/affinity-groups versions = all
27222025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/affinity-groups versions = all
27232025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group} versions = all
27242025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/affinity-groups/{affinity_group} versions = all
27252025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/affinity-groups/{affinity_group} versions = all
27262025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members versions = all
27272025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
27282025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
27292025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/affinity-groups/{affinity_group}/members/instance/{instance} versions = all
27302025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/anti-affinity-groups versions = all
27312025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/anti-affinity-groups versions = all
27322025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
27332025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
27342025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/anti-affinity-groups/{anti_affinity_group} versions = all
27352025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members versions = all
27362025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
27372025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
27382025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/anti-affinity-groups/{anti_affinity_group}/members/instance/{instance} versions = all
27392025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/certificates versions = all
27402025-04-30T06:13:10.666ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/certificates versions = all
27412025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/certificates/{certificate} versions = all
27422025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/certificates/{certificate} versions = all
27432025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/disks versions = all
27442025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/disks versions = all
27452025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/disks/{disk} versions = all
27462025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/disks/{disk} versions = all
27472025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/disks/{disk}/bulk-write versions = all
27482025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/disks/{disk}/bulk-write-start versions = all
27492025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/disks/{disk}/bulk-write-stop versions = all
27502025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/disks/{disk}/finalize versions = all
27512025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/disks/{disk}/metrics/{metric} versions = all
27522025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/floating-ips versions = all
27532025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/floating-ips versions = all
27542025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/floating-ips/{floating_ip} versions = all
27552025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/floating-ips/{floating_ip} versions = all
27562025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/floating-ips/{floating_ip} versions = all
27572025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/floating-ips/{floating_ip}/attach versions = all
27582025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/floating-ips/{floating_ip}/detach versions = all
27592025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/groups versions = all
27602025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/groups/{group_id} versions = all
27612025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/images versions = all
27622025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/images versions = all
27632025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/images/{image} versions = all
27642025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/images/{image} versions = all
27652025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/images/{image}/demote versions = all
27662025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/images/{image}/promote versions = all
27672025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/instances versions = all
27682025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/instances versions = all
27692025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/instances/{instance} versions = all
27702025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/instances/{instance} versions = all
27712025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/instances/{instance} versions = all
27722025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/instances/{instance}/affinity-groups versions = all
27732025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/instances/{instance}/anti-affinity-groups versions = all
27742025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/instances/{instance}/disks versions = all
27752025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/instances/{instance}/disks/attach versions = all
27762025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/instances/{instance}/disks/detach versions = all
27772025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/instances/{instance}/external-ips versions = all
27782025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27792025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/instances/{instance}/external-ips/ephemeral versions = all
27802025-04-30T06:13:10.667ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/instances/{instance}/reboot versions = all
27812025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/instances/{instance}/serial-console versions = all
27822025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/instances/{instance}/serial-console/stream versions = all
27832025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/instances/{instance}/ssh-public-keys versions = all
27842025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/instances/{instance}/start versions = all
27852025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/instances/{instance}/stop versions = all
27862025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/internet-gateway-ip-addresses versions = all
27872025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/internet-gateway-ip-addresses versions = all
27882025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/internet-gateway-ip-addresses/{address} versions = all
27892025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/internet-gateway-ip-pools versions = all
27902025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/internet-gateway-ip-pools versions = all
27912025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/internet-gateway-ip-pools/{pool} versions = all
27922025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/internet-gateways versions = all
27932025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/internet-gateways versions = all
27942025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/internet-gateways/{gateway} versions = all
27952025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/internet-gateways/{gateway} versions = all
27962025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/ip-pools versions = all
27972025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/ip-pools/{pool} versions = all
27982025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/login/{silo_name}/local versions = all
27992025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/logout versions = all
28002025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/me versions = all
28012025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/me/groups versions = all
28022025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/me/ssh-keys versions = all
28032025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/me/ssh-keys versions = all
28042025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/me/ssh-keys/{ssh_key} versions = all
28052025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/me/ssh-keys/{ssh_key} versions = all
28062025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/metrics/{metric_name} versions = all
28072025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/network-interfaces versions = all
28082025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/network-interfaces versions = all
28092025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/network-interfaces/{interface} versions = all
28102025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/network-interfaces/{interface} versions = all
28112025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/network-interfaces/{interface} versions = all
28122025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/ping versions = all
28132025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/policy versions = all
28142025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/policy versions = all
28152025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/projects versions = all
28162025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/projects versions = all
28172025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/projects/{project} versions = all
28182025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/projects/{project} versions = all
28192025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/projects/{project} versions = all
28202025-04-30T06:13:10.668ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/projects/{project}/policy versions = all
28212025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/projects/{project}/policy versions = all
28222025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/snapshots versions = all
28232025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/snapshots versions = all
28242025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/snapshots/{snapshot} versions = all
28252025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/snapshots/{snapshot} versions = all
28262025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/disks versions = all
28272025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/disks/{disk_id} versions = all
28282025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/rack-switch-port/{rack_id}/{switch_location}/{port}/lldp/neighbors versions = all
28292025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/racks versions = all
28302025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/racks/{rack_id} versions = all
28312025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/sleds versions = all
28322025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/hardware/sleds versions = all
28332025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id} versions = all
28342025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/disks versions = all
28352025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/sleds/{sled_id}/instances versions = all
28362025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/hardware/sleds/{sled_id}/provision-policy versions = all
28372025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/sleds-uninitialized versions = all
28382025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/switch-port versions = all
28392025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
28402025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/lldp/config versions = all
28412025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/hardware/switch-port/{port}/settings versions = all
28422025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/hardware/switch-port/{port}/settings versions = all
28432025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/switch-port/{port}/status versions = all
28442025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/switches versions = all
28452025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/hardware/switches/{switch_id} versions = all
28462025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/identity-providers versions = all
28472025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/identity-providers/local/users versions = all
28482025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/identity-providers/local/users/{user_id} versions = all
28492025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/identity-providers/local/users/{user_id}/set-password versions = all
28502025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/identity-providers/saml versions = all
28512025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/identity-providers/saml/{provider} versions = all
28522025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/ip-pools versions = all
28532025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/ip-pools versions = all
28542025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/ip-pools/{pool} versions = all
28552025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/ip-pools/{pool} versions = all
28562025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/ip-pools/{pool} versions = all
28572025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/ip-pools/{pool}/ranges versions = all
28582025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/add versions = all
28592025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/ip-pools/{pool}/ranges/remove versions = all
28602025-04-30T06:13:10.669ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/ip-pools/{pool}/silos versions = all
28612025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/ip-pools/{pool}/silos versions = all
28622025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28632025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/ip-pools/{pool}/silos/{silo} versions = all
28642025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/ip-pools/{pool}/utilization versions = all
28652025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/ip-pools-service versions = all
28662025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/ip-pools-service/ranges versions = all
28672025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/ip-pools-service/ranges/add versions = all
28682025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/ip-pools-service/ranges/remove versions = all
28692025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/metrics/{metric_name} versions = all
28702025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/address-lot versions = all
28712025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/networking/address-lot versions = all
28722025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/networking/address-lot/{address_lot} versions = all
28732025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/address-lot/{address_lot}/blocks versions = all
28742025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/allow-list versions = all
28752025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/networking/allow-list versions = all
28762025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/networking/bfd-disable versions = all
28772025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/networking/bfd-enable versions = all
28782025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/bfd-status versions = all
28792025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/networking/bgp versions = all
28802025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/bgp versions = all
28812025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/networking/bgp versions = all
28822025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/bgp-announce-set versions = all
28832025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/networking/bgp-announce-set versions = all
28842025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/networking/bgp-announce-set/{announce_set} versions = all
28852025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/bgp-announce-set/{announce_set}/announcement versions = all
28862025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/bgp-exported versions = all
28872025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/bgp-message-history versions = all
28882025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/bgp-routes-ipv4 versions = all
28892025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/bgp-status versions = all
28902025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/loopback-address versions = all
28912025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/networking/loopback-address versions = all
28922025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/networking/loopback-address/{rack_id}/{switch_location}/{address}/{subnet_mask} versions = all
28932025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/networking/switch-port-settings versions = all
28942025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/switch-port-settings versions = all
28952025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/networking/switch-port-settings versions = all
28962025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/networking/switch-port-settings/{port} versions = all
28972025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/policy versions = all
28982025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/policy versions = all
28992025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/roles versions = all
29002025-04-30T06:13:10.670ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/roles/{role_name} versions = all
29012025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/silo-quotas versions = all
29022025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/silos versions = all
29032025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/silos versions = all
29042025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/system/silos/{silo} versions = all
29052025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/silos/{silo} versions = all
29062025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/silos/{silo}/ip-pools versions = all
29072025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/silos/{silo}/policy versions = all
29082025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/silos/{silo}/policy versions = all
29092025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/silos/{silo}/quotas versions = all
29102025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/silos/{silo}/quotas versions = all
29112025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/system/timeseries/query versions = all
29122025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/timeseries/schemas versions = all
29132025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/update/repository versions = all
29142025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/update/repository/{system_version} versions = all
29152025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/update/target-release versions = all
29162025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/system/update/target-release versions = all
29172025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/users versions = all
29182025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/users/{user_id} versions = all
29192025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/users-builtin versions = all
29202025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/users-builtin/{user} versions = all
29212025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/utilization/silos versions = all
29222025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/system/utilization/silos/{silo} versions = all
29232025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/timeseries/query versions = all
29242025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/users versions = all
29252025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/utilization versions = all
29262025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpc-firewall-rules versions = all
29272025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/vpc-firewall-rules versions = all
29282025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpc-router-routes versions = all
29292025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/vpc-router-routes versions = all
29302025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/vpc-router-routes/{route} versions = all
29312025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpc-router-routes/{route} versions = all
29322025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/vpc-router-routes/{route} versions = all
29332025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpc-routers versions = all
29342025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/vpc-routers versions = all
29352025-04-30T06:13:10.671ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/vpc-routers/{router} versions = all
29362025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpc-routers/{router} versions = all
29372025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/vpc-routers/{router} versions = all
29382025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpc-subnets versions = all
29392025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/vpc-subnets versions = all
29402025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/vpc-subnets/{subnet} versions = all
29412025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpc-subnets/{subnet} versions = all
29422025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/vpc-subnets/{subnet} versions = all
29432025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpc-subnets/{subnet}/network-interfaces versions = all
29442025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpcs versions = all
29452025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/vpcs versions = all
29462025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/vpcs/{vpc} versions = all
29472025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/vpcs/{vpc} versions = all
29482025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/vpcs/{vpc} versions = all
29492025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/webhooks/deliveries versions = all
29502025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/webhooks/deliveries/{event_id}/resend versions = all
29512025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/webhooks/event-classes versions = all
29522025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/webhooks/receivers versions = all
29532025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/webhooks/receivers versions = all
29542025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/webhooks/receivers/{receiver} versions = all
29552025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/webhooks/receivers/{receiver} versions = all
29562025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = PUT path = /v1/webhooks/receivers/{receiver} versions = all
29572025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/webhooks/receivers/{receiver}/probe versions = all
29582025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/webhooks/receivers/{receiver}/subscriptions versions = all
29592025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/webhooks/receivers/{receiver}/subscriptions/{subscription} versions = all
29602025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = GET path = /v1/webhooks/secrets versions = all
29612025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = POST path = /v1/webhooks/secrets versions = all
29622025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): registered endpoint local_addr = [::1]:12228 method = DELETE path = /v1/webhooks/secrets/{secret_id} versions = all
29632025-04-30T06:13:10.672ZINFOomicron-dev (dropshot_external_techport): listening local_addr = [::1]:12228
29642025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot_external_techport): successfully registered DTrace USDT probes local_addr = [::1]:12228
29652025-04-30T06:13:10.672ZDEBGomicron-dev (nexus-producer-server): registered DTrace probes
29662025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:50290 method = GET path = /{producer_id} versions = all
29672025-04-30T06:13:10.672ZINFOomicron-dev (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = [::1]:50290
29682025-04-30T06:13:10.672ZDEBGomicron-dev (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:50290
29692025-04-30T06:13:10.672ZDEBGomicron-dev (nexus-producer-server): Requested any available port, Dropshot server has been bound to [::1]:50290
29702025-04-30T06:13:10.672ZDEBGomicron-dev (nexus-producer-server): Nexus IP provided explicitly, will use it addr = [::1]:12221
29712025-04-30T06:13:10.672ZDEBGomicron-dev (nexus-producer-server): starting producer registration task
29722025-04-30T06:13:10.672ZINFOomicron-dev (nexus-producer-server): starting oximeter metric producer server address = [::1]:50290 file = oximeter/producer/src/lib.rs:279 interval = 10s producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
29732025-04-30T06:13:10.672ZDEBGomicron-dev (producer-registration-task): registering as oximeter producer with Nexus registration_count = 0
29742025-04-30T06:13:10.673ZDEBGomicron-dev (producer-registration-task): will register with Nexus at [::1]:12221
29752025-04-30T06:13:10.673ZDEBGomicron-dev (init_with_steps): Running step start_oximeter
29762025-04-30T06:13:10.673ZDEBGomicron-dev: Starting Oximeter
29772025-04-30T06:13:10.673ZDEBGomicron-dev (oximeter): registered DTrace probes
29782025-04-30T06:13:10.673ZINFOomicron-dev (oximeter): starting oximeter server file = oximeter/collector/src/lib.rs:236
29792025-04-30T06:13:10.673ZDEBGomicron-dev (oximeter): creating ClickHouse client
29802025-04-30T06:13:10.675ZDEBGomicron-dev (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
29812025-04-30T06:13:10.679ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = SELECT MAX(value) AS max_version FROM oximeter.version;
29822025-04-30T06:13:10.700ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet
29832025-04-30T06:13:10.700ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_rendezvous resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
29842025-04-30T06:13:10.701ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database
29852025-04-30T06:13:10.701ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
29862025-04-30T06:13:10.701ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
29872025-04-30T06:13:10.702ZWARNomicron-dev (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:894 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
29882025-04-30T06:13:10.702ZDEBGomicron-dev (oximeter-agent): oximeter database does not exist, creating collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
29892025-04-30T06:13:10.702ZINFOomicron-dev (clickhouse-client): reading db version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:794 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
29902025-04-30T06:13:10.706ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 112.358788ms iteration = 2 status = {"physical_disks_added":0}
29912025-04-30T06:13:10.708ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = SELECT MAX(value) AS max_version FROM oximeter.version;
29922025-04-30T06:13:10.709ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
29932025-04-30T06:13:10.711ZWARNomicron-dev (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:894 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
29942025-04-30T06:13:10.711ZINFOomicron-dev (clickhouse-client): read oximeter database version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:799 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e version = 0
29952025-04-30T06:13:10.711ZINFOomicron-dev (clickhouse-client): initializing oximeter schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:804 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
29962025-04-30T06:13:10.712ZDEBGomicron-dev (clickhouse-client): initializing ClickHouse database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
29972025-04-30T06:13:10.713ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = CREATE DATABASE IF NOT EXISTS oximeter
29982025-04-30T06:13:10.714ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
29992025-04-30T06:13:10.715ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet
30002025-04-30T06:13:10.715ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_rendezvous resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30012025-04-30T06:13:10.718ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database
30022025-04-30T06:13:10.718ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
30032025-04-30T06:13:10.718ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
30042025-04-30T06:13:10.718ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\n/* The version table contains metadata about the `oximeter` database */\nCREATE TABLE IF NOT EXISTS oximeter.version\n(\n value UInt64,\n timestamp DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (value, timestamp)
30052025-04-30T06:13:10.721ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30062025-04-30T06:13:10.722ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\n/* The measurement tables contain all individual samples from each timeseries.\n *\n * Each table stores a single datum type, and otherwise contains nearly the same\n * structure. The primary sorting key is on the timeseries name, key, and then\n * timestamp, so that all timeseries from the same schema are grouped, followed\n * by all samples from the same timeseries.\n *\n * This reflects that one usually looks up the _key_ in one or more field table,\n * and then uses that to index quickly into the measurements tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Bool)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30072025-04-30T06:13:10.734ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
30082025-04-30T06:13:10.734ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database
30092025-04-30T06:13:10.735ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
30102025-04-30T06:13:10.735ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
30112025-04-30T06:13:10.736ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30122025-04-30T06:13:10.742ZDEBGomicron-dev (producer-registration-task): client request body = Some(Body) method = POST uri = http://[::1]:12221/metrics/producers
30132025-04-30T06:13:10.742ZINFOomicron-dev (dropshot_internal): accepted connection local_addr = [::1]:12221 remote_addr = [::1]:35790
30142025-04-30T06:13:10.743ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = POST remote_addr = [::1]:35790 req_id = 1c677ec9-7442-4788-ab77-382b54cdcb88 uri = /metrics/producers
30152025-04-30T06:13:10.745ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:35790 req_id = 1c677ec9-7442-4788-ab77-382b54cdcb88 resource = Database uri = /metrics/producers
30162025-04-30T06:13:10.745ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:35790 req_id = 1c677ec9-7442-4788-ab77-382b54cdcb88 roles = RoleSet { roles: {} } uri = /metrics/producers
30172025-04-30T06:13:10.745ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:35790 req_id = 1c677ec9-7442-4788-ab77-382b54cdcb88 resource = Database result = Ok(()) uri = /metrics/producers
30182025-04-30T06:13:10.746ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30192025-04-30T06:13:10.752ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30202025-04-30T06:13:10.758ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet
30212025-04-30T06:13:10.759ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_rendezvous resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30222025-04-30T06:13:10.759ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database
30232025-04-30T06:13:10.759ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
30242025-04-30T06:13:10.760ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
30252025-04-30T06:13:10.762ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30262025-04-30T06:13:10.770ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
30272025-04-30T06:13:10.771ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet
30282025-04-30T06:13:10.771ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_rendezvous resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
30292025-04-30T06:13:10.771ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database
30302025-04-30T06:13:10.771ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
30312025-04-30T06:13:10.771ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
30322025-04-30T06:13:10.810ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt16)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30332025-04-30T06:13:10.815ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30342025-04-30T06:13:10.816ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
30352025-04-30T06:13:10.820ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30362025-04-30T06:13:10.822ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Fleet result = Ok(())
30372025-04-30T06:13:10.822ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database
30382025-04-30T06:13:10.822ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_rendezvous roles = RoleSet { roles: {} }
30392025-04-30T06:13:10.823ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_rendezvous resource = Database result = Ok(())
30402025-04-30T06:13:10.823ZINFOomicron-dev (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 80597 local_addr = [::1]:12221 method = POST remote_addr = [::1]:35790 req_id = 1c677ec9-7442-4788-ab77-382b54cdcb88 response_code = 503 uri = /metrics/producers
30412025-04-30T06:13:10.824ZDEBGomicron-dev (producer-registration-task): client response result = Ok(Response { url: "http://[::1]:12221/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "1c677ec9-7442-4788-ab77-382b54cdcb88", "content-length": "133", "date": "Wed, 30 Apr 2025 06:13:10 GMT"} })
30422025-04-30T06:13:10.826ZWARNomicron-dev (producer-registration-task): failed to register with Nexus, will retry count = 0 delay = 154.255995ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"1c677ec9-7442-4788-ab77-382b54cdcb88\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:13:10 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"1c677ec9-7442-4788-ab77-382b54cdcb88\\" }" file = oximeter/producer/src/lib.rs:371
30432025-04-30T06:13:10.827ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30442025-04-30T06:13:10.830ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30452025-04-30T06:13:10.831ZINFOomicron-dev (BackgroundTasks): ensured all Crucible datasets present in inventory have database records background_task = blueprint_rendezvous num_already_exist = 0 num_inserted = 0 num_not_in_inventory = 0
30462025-04-30T06:13:10.831ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = blueprint_rendezvous elapsed = 238.244106ms iteration = 2 status = {"blueprint_id":"1bfc4639-5ca6-4323-a678-d4a0347a6fdf","inventory_collection_id":"56c2f67f-67f1-43f2-8f3d-f724f719303e","stats":{"crucible_dataset":{"num_already_exist":0,"num_inserted":0,"num_not_in_inventory":0},"debug_dataset":{"num_already_exist":0,"num_already_tombstoned":0,"num_inserted":0,"num_not_in_inventory":0,"num_tombstoned":0}}}
30472025-04-30T06:13:10.834ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f32\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30482025-04-30T06:13:10.838ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_f64\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30492025-04-30T06:13:10.843ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(String)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30502025-04-30T06:13:10.848ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_bytes\n(\n timeseries_name String,\n timeseries_key UInt64,\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Right now we can't unambiguously record a nullable byte array.\n * Arrays cannot be nested in `Nullable()` types, and encoding the array as\n * a string isn't palatable for a few reasons.\n * See: https://github.com/oxidecomputer/omicron/issues/4551 for more\n * details.\n */\n datum Array(UInt8)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30512025-04-30T06:13:10.851ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativei64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Int64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30522025-04-30T06:13:10.855ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativeu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(UInt64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30532025-04-30T06:13:10.861ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float32)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30542025-04-30T06:13:10.864ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_cumulativef64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n datum Nullable(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30552025-04-30T06:13:10.868ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n /*\n * NOTE: Array types cannot be Nullable, see\n * https://clickhouse.com/docs/en/sql-reference/data-types/nullable\n * for more details.\n *\n * This means we need to use empty arrays to indicate a missing value. This\n * is unfortunate, and at this point relies on the fact that an\n * `oximeter::Histogram` cannot have zero bins. If that changes, we'll need\n * to figure out another way to represent missing samples here.\n */\n bins Array(Int8),\n counts Array(UInt64),\n min Int8,\n max Int8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30562025-04-30T06:13:10.879ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu8\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt8),\n counts Array(UInt64),\n min UInt8,\n max UInt8,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30572025-04-30T06:13:10.886ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int16),\n counts Array(UInt64),\n min Int16,\n max Int16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30582025-04-30T06:13:10.890ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu16\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt16),\n counts Array(UInt64),\n min UInt16,\n max UInt16,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30592025-04-30T06:13:10.907ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int32),\n counts Array(UInt64),\n min Int32,\n max Int32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30602025-04-30T06:13:10.913ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt32),\n counts Array(UInt64),\n min UInt32,\n max UInt32,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30612025-04-30T06:13:10.920ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogrami64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Int64),\n counts Array(UInt64),\n min Int64,\n max Int64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30622025-04-30T06:13:10.924ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramu64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(UInt64),\n counts Array(UInt64),\n min UInt64,\n max UInt64,\n sum_of_samples Int64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30632025-04-30T06:13:10.935ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf32\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float32),\n counts Array(UInt64),\n min Float32,\n max Float32,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30642025-04-30T06:13:10.942ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.measurements_histogramf64\n(\n timeseries_name String,\n timeseries_key UInt64,\n start_time DateTime64(9, 'UTC'),\n timestamp DateTime64(9, 'UTC'),\n bins Array(Float64),\n counts Array(UInt64),\n min Float64,\n max Float64,\n sum_of_samples Float64,\n squared_mean Float64,\n p50_marker_heights Array(Float64),\n p50_marker_positions Array(UInt64),\n p50_desired_marker_positions Array(Float64),\n p90_marker_heights Array(Float64),\n p90_marker_positions Array(UInt64),\n p90_desired_marker_positions Array(Float64),\n p99_marker_heights Array(Float64),\n p99_marker_positions Array(UInt64),\n p99_desired_marker_positions Array(Float64)\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, timeseries_key, start_time, timestamp)\nTTL toDateTime(timestamp) + INTERVAL 30 DAY
30652025-04-30T06:13:10.947ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\n/* The field tables store named dimensions of each timeseries.\n *\n * As with the measurement tables, there is one field table for each field data\n * type. Fields are deduplicated by using the "replacing merge tree", though\n * this behavior **must not** be relied upon for query correctness.\n *\n * The index for the fields differs from the measurements, however. Rows are\n * sorted by timeseries name, then field name, field value, and finally\n * timeseries key. This reflects the most common pattern for looking them up:\n * by field name and possibly value, within a timeseries. The resulting keys are\n * usually then used to look up measurements.\n *\n * NOTE: We may want to consider a secondary index on these tables, sorting by\n * timeseries name and then key, since it would improve lookups where one\n * already has the key. Realistically though, these tables are quite small and\n * so performance benefits will be low in absolute terms.\n *\n * TTL: We use a materialized column to expire old field table records. This\n * column is generated automatically by the database whenever a new row is\n * inserted. It cannot be inserted directly, nor is it returned in a `SELECT *`\n * query. Since these tables are `ReplacingMergeTree`s, that means the last\n * record will remain during a deduplication, which will have the last\n * timestamp. ClickHouse will then expire old data for us, similar to the\n * measurement tables.\n */\nCREATE TABLE IF NOT EXISTS oximeter.fields_bool\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Bool,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30662025-04-30T06:13:10.953ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30672025-04-30T06:13:10.961ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u8\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt8,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30682025-04-30T06:13:10.968ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30692025-04-30T06:13:10.978ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u16\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt16,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30702025-04-30T06:13:10.980ZDEBGomicron-dev (producer-registration-task): will register with Nexus at [::1]:12221
30712025-04-30T06:13:11.019ZDEBGomicron-dev (producer-registration-task): client request body = Some(Body) method = POST uri = http://[::1]:12221/metrics/producers
30722025-04-30T06:13:11.020ZINFOomicron-dev (dropshot_internal): accepted connection local_addr = [::1]:12221 remote_addr = [::1]:52312
30732025-04-30T06:13:11.020ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30742025-04-30T06:13:11.021ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = POST remote_addr = [::1]:52312 req_id = 6ad8072c-4c4d-42d1-b7c3-789fe9e83550 uri = /metrics/producers
30752025-04-30T06:13:11.021ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:52312 req_id = 6ad8072c-4c4d-42d1-b7c3-789fe9e83550 resource = Database uri = /metrics/producers
30762025-04-30T06:13:11.021ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:52312 req_id = 6ad8072c-4c4d-42d1-b7c3-789fe9e83550 roles = RoleSet { roles: {} } uri = /metrics/producers
30772025-04-30T06:13:11.022ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:52312 req_id = 6ad8072c-4c4d-42d1-b7c3-789fe9e83550 resource = Database result = Ok(()) uri = /metrics/producers
30782025-04-30T06:13:11.025ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u32\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt32,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30792025-04-30T06:13:11.028ZINFOomicron-dev (dropshot_internal): request completed error_message_external = Service Unavailable error_message_internal = no Oximeter instances available for assignment latency_us = 7755 local_addr = [::1]:12221 method = POST remote_addr = [::1]:52312 req_id = 6ad8072c-4c4d-42d1-b7c3-789fe9e83550 response_code = 503 uri = /metrics/producers
30802025-04-30T06:13:11.029ZDEBGomicron-dev (producer-registration-task): client response result = Ok(Response { url: "http://[::1]:12221/metrics/producers", status: 503, headers: {"content-type": "application/json", "x-request-id": "6ad8072c-4c4d-42d1-b7c3-789fe9e83550", "content-length": "133", "date": "Wed, 30 Apr 2025 06:13:10 GMT"} })
30812025-04-30T06:13:11.031ZWARNomicron-dev (producer-registration-task): failed to register with Nexus, will retry count = 1 delay = 359.325142ms error = "Error Response: status: 503 Service Unavailable; headers: {\\"content-type\\": \\"application/json\\", \\"x-request-id\\": \\"6ad8072c-4c4d-42d1-b7c3-789fe9e83550\\", \\"content-length\\": \\"133\\", \\"date\\": \\"Wed, 30 Apr 2025 06:13:10 GMT\\"}; value: Error { error_code: Some(\\"ServiceNotAvailable\\"), message: \\"Service Unavailable\\", request_id: \\"6ad8072c-4c4d-42d1-b7c3-789fe9e83550\\" }" file = oximeter/producer/src/lib.rs:371
30822025-04-30T06:13:11.031ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_i64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value Int64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30832025-04-30T06:13:11.036ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_u64\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UInt64,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30842025-04-30T06:13:11.045ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_ipaddr\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value IPv6,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30852025-04-30T06:13:11.050ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_string\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value String,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30862025-04-30T06:13:11.059ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\nCREATE TABLE IF NOT EXISTS oximeter.fields_uuid\n(\n timeseries_name String,\n timeseries_key UInt64,\n field_name String,\n field_value UUID,\n last_updated_at DateTime MATERIALIZED now()\n)\nENGINE = ReplacingMergeTree()\nORDER BY (timeseries_name, field_name, field_value, timeseries_key)\nTTL last_updated_at + INTERVAL 30 DAY
30872025-04-30T06:13:11.070ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = \n\n/* The timeseries schema table stores the extracted schema for the samples\n * oximeter collects.\n */\nCREATE TABLE IF NOT EXISTS oximeter.timeseries_schema\n(\n timeseries_name String,\n fields Nested(\n name String,\n type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'IpAddr' = 3,\n 'String' = 4,\n 'Uuid' = 6,\n 'I8' = 7,\n 'U8' = 8,\n 'I16' = 9,\n 'U16' = 10,\n 'I32' = 11,\n 'U32' = 12,\n 'U64' = 13\n ),\n source Enum(\n 'Target' = 1,\n 'Metric' = 2\n )\n ),\n datum_type Enum(\n 'Bool' = 1,\n 'I64' = 2,\n 'F64' = 3,\n 'String' = 4,\n 'Bytes' = 5,\n 'CumulativeI64' = 6,\n 'CumulativeF64' = 7,\n 'HistogramI64' = 8,\n 'HistogramF64' = 9,\n 'I8' = 10,\n 'U8' = 11,\n 'I16' = 12,\n 'U16' = 13,\n 'I32' = 14,\n 'U32' = 15,\n 'U64' = 16,\n 'F32' = 17,\n 'CumulativeU64' = 18,\n 'CumulativeF32' = 19,\n 'HistogramI8' = 20,\n 'HistogramU8' = 21,\n 'HistogramI16' = 22,\n 'HistogramU16' = 23,\n 'HistogramI32' = 24,\n 'HistogramU32' = 25,\n 'HistogramU64' = 26,\n 'HistogramF32' = 27\n ),\n created DateTime64(9, 'UTC')\n)\nENGINE = MergeTree()\nORDER BY (timeseries_name, fields.name)
30882025-04-30T06:13:11.079ZINFOomicron-dev (clickhouse-client): inserting current version collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/db/src/client/mod.rs:825 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e version = 14
30892025-04-30T06:13:11.079ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = INSERT INTO oximeter.version (*) VALUES (14, now());
30902025-04-30T06:13:11.089ZDEBGomicron-dev (clickhouse-client): registered USDT probes collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 12d46c38-55ff-40e0-9eec-b1ca7e545d08
30912025-04-30T06:13:11.092ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:41936 method = GET path = /info versions = all
30922025-04-30T06:13:11.092ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:41936 method = GET path = /producers versions = all
30932025-04-30T06:13:11.092ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:41936 method = DELETE path = /producers/{producer_id} versions = all
30942025-04-30T06:13:11.092ZDEBGomicron-dev (dropshot): registered endpoint local_addr = [::1]:41936 method = GET path = /producers/{producer_id} versions = all
30952025-04-30T06:13:11.092ZINFOomicron-dev (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:328 local_addr = [::1]:41936
30962025-04-30T06:13:11.093ZDEBGomicron-dev (dropshot): successfully registered DTrace USDT probes local_addr = [::1]:41936
30972025-04-30T06:13:11.095ZDEBGomicron-dev (oximeter): registered USDT probes
30982025-04-30T06:13:11.095ZDEBGomicron-dev (oximeter): contacting nexus
30992025-04-30T06:13:11.188ZDEBGomicron-dev (oximeter): client request body = Some(Body) method = POST uri = http://[::1]:12221/metrics/collectors
31002025-04-30T06:13:12.587ZINFOomicron-dev (dropshot_internal): accepted connection local_addr = [::1]:12221 remote_addr = [::1]:40743
31012025-04-30T06:13:12.590ZDEBGomicron-dev (producer-registration-task): will register with Nexus at [::1]:12221
31022025-04-30T06:13:12.657ZTRAComicron-dev (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31032025-04-30T06:13:12.657ZTRAComicron-dev (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
31042025-04-30T06:13:12.658ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = POST remote_addr = [::1]:40743 req_id = 36b67141-cac4-42b8-8e34-7397741b40f4 uri = /metrics/collectors
31052025-04-30T06:13:12.659ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:40743 req_id = 36b67141-cac4-42b8-8e34-7397741b40f4 resource = Database uri = /metrics/collectors
31062025-04-30T06:13:12.659ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:40743 req_id = 36b67141-cac4-42b8-8e34-7397741b40f4 roles = RoleSet { roles: {} } uri = /metrics/collectors
31072025-04-30T06:13:12.660ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:40743 req_id = 36b67141-cac4-42b8-8e34-7397741b40f4 resource = Database result = Ok(()) uri = /metrics/collectors
31082025-04-30T06:13:12.671ZINFOomicron-dev (nexus): registered new oximeter metric collection server address = [::1]:41936 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31092025-04-30T06:13:12.671ZINFOomicron-dev (dropshot_internal): request completed latency_us = 12375 local_addr = [::1]:12221 method = POST remote_addr = [::1]:40743 req_id = 36b67141-cac4-42b8-8e34-7397741b40f4 response_code = 204 uri = /metrics/collectors
31102025-04-30T06:13:12.676ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors", status: 204, headers: {"x-request-id": "36b67141-cac4-42b8-8e34-7397741b40f4", "date": "Wed, 30 Apr 2025 06:13:12 GMT"} })
31112025-04-30T06:13:12.676ZINFOomicron-dev (oximeter): oximeter registered with nexus file = oximeter/collector/src/lib.rs:392 id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78
31122025-04-30T06:13:12.676ZDEBGomicron-dev (init_with_steps): Running step start_producer_server
31132025-04-30T06:13:12.676ZDEBGomicron-dev: Starting test metric Producer Server
31142025-04-30T06:13:12.678ZINFOomicron-dev (oximeter-agent): starting refresh list task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:559
31152025-04-30T06:13:12.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
31162025-04-30T06:13:12.680ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34446 method = GET path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31172025-04-30T06:13:12.680ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34446 method = POST path = /crucible/0/regions server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31182025-04-30T06:13:12.680ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34446 method = DELETE path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31192025-04-30T06:13:12.680ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34446 method = GET path = /crucible/0/regions/{id} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31202025-04-30T06:13:12.680ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34446 method = GET path = /crucible/0/regions/{id}/snapshots server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31212025-04-30T06:13:12.680ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34446 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31222025-04-30T06:13:12.680ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34446 method = GET path = /crucible/0/regions/{id}/snapshots/{name} server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31232025-04-30T06:13:12.680ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34446 method = DELETE path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31242025-04-30T06:13:12.681ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): registered endpoint kind = storage local_addr = [::1]:34446 method = POST path = /crucible/0/regions/{id}/snapshots/{name}/run server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 versions = all
31252025-04-30T06:13:12.681ZINFOomicron-dev (Simulated CrucibleAgent Dropshot Server): listening kind = storage local_addr = [::1]:34446 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
31262025-04-30T06:13:12.681ZDEBGomicron-dev (Simulated CrucibleAgent Dropshot Server): successfully registered DTrace USDT probes kind = storage local_addr = [::1]:34446 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
31272025-04-30T06:13:12.681ZINFOomicron-dev (SledAgent): Created Simulated Crucible Server address = [::1]:34446 kind = storage server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
31282025-04-30T06:13:12.681ZINFOomicron-dev (nexus): inserting test physical disk model = test-model physical_disk_id = a6e7b196-0e87-4524-91e0-b5abebea168d serial = totally-unique-serial: a6e7b196-0e87-4524-91e0-b5abebea168d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vendor = test-vendor
31292025-04-30T06:13:12.681ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31302025-04-30T06:13:12.681ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31312025-04-30T06:13:12.682ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31322025-04-30T06:13:12.698ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31332025-04-30T06:13:12.698ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6558aff9-dec9-4da4-bd55-27124956c7d0 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31342025-04-30T06:13:12.702ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6558aff9-dec9-4da4-bd55-27124956c7d0 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31352025-04-30T06:13:12.702ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6558aff9-dec9-4da4-bd55-27124956c7d0 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31362025-04-30T06:13:12.703ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6558aff9-dec9-4da4-bd55-27124956c7d0 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31372025-04-30T06:13:12.707ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31382025-04-30T06:13:12.707ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31392025-04-30T06:13:12.708ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31402025-04-30T06:13:12.711ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
31412025-04-30T06:13:12.712ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31422025-04-30T06:13:12.712ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31432025-04-30T06:13:12.712ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31442025-04-30T06:13:12.712ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31452025-04-30T06:13:12.715ZINFOomicron-dev (dropshot_internal): request completed latency_us = 16366 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6558aff9-dec9-4da4-bd55-27124956c7d0 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
31462025-04-30T06:13:12.716ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "6558aff9-dec9-4da4-bd55-27124956c7d0", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:12 GMT"} })
31472025-04-30T06:13:12.717ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 0 n_pruned_tasks = 0
31482025-04-30T06:13:12.718ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31492025-04-30T06:13:12.722ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
31502025-04-30T06:13:12.818ZINFOomicron-dev (nexus): upserting zpool physical_disk_id = a6e7b196-0e87-4524-91e0-b5abebea168d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 zpool_id = 9102ae2a-61d0-4533-8047-8c21995e5c4c
31512025-04-30T06:13:12.818ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31522025-04-30T06:13:12.818ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31532025-04-30T06:13:12.819ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31542025-04-30T06:13:12.820ZTRAComicron-dev (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
31552025-04-30T06:13:12.820ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31562025-04-30T06:13:12.820ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31572025-04-30T06:13:12.820ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31582025-04-30T06:13:12.821ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31592025-04-30T06:13:12.824ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31602025-04-30T06:13:12.828ZDEBGomicron-dev (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
31612025-04-30T06:13:12.837ZINFOomicron-dev (nexus): upserting Crucible dataset dataset_id = 95b80f45-5df0-4209-b416-1dffbcf3ed78 zpool_id = 9102ae2a-61d0-4533-8047-8c21995e5c4c
31622025-04-30T06:13:12.848ZINFOomicron-dev (nexus): inserting test physical disk model = test-model physical_disk_id = a6e7b196-0e87-4524-91e0-b5abebea168d serial = totally-unique-serial: a6e7b196-0e87-4524-91e0-b5abebea168d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 vendor = test-vendor
31632025-04-30T06:13:12.848ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31642025-04-30T06:13:12.849ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31652025-04-30T06:13:12.849ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31662025-04-30T06:13:12.855ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = PhysicalDisk { parent: Fleet, key: a6e7b196-0e87-4524-91e0-b5abebea168d (physical_disk), lookup_type: ById(a6e7b196-0e87-4524-91e0-b5abebea168d) }
31672025-04-30T06:13:12.855ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31682025-04-30T06:13:12.855ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31692025-04-30T06:13:12.855ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31702025-04-30T06:13:12.855ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31712025-04-30T06:13:12.858ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31722025-04-30T06:13:12.863ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = PhysicalDisk { parent: Fleet, key: a6e7b196-0e87-4524-91e0-b5abebea168d (physical_disk), lookup_type: ById(a6e7b196-0e87-4524-91e0-b5abebea168d) } result = Ok(())
31732025-04-30T06:13:12.863ZINFOomicron-dev (nexus): upserting zpool physical_disk_id = a6e7b196-0e87-4524-91e0-b5abebea168d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 zpool_id = 9102ae2a-61d0-4533-8047-8c21995e5c4c
31742025-04-30T06:13:12.863ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31752025-04-30T06:13:12.863ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31762025-04-30T06:13:12.864ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31772025-04-30T06:13:12.870ZTRAComicron-dev (InternalApi): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
31782025-04-30T06:13:12.870ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31792025-04-30T06:13:12.871ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31802025-04-30T06:13:12.871ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31812025-04-30T06:13:12.871ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31822025-04-30T06:13:12.875ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31832025-04-30T06:13:12.878ZDEBGomicron-dev (InternalApi): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
31842025-04-30T06:13:12.885ZTRAComicron-dev (InternalApi): authorize begin action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet
31852025-04-30T06:13:12.885ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31862025-04-30T06:13:12.885ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31872025-04-30T06:13:12.885ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31882025-04-30T06:13:12.886ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31892025-04-30T06:13:12.893ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
31902025-04-30T06:13:12.898ZDEBGomicron-dev (InternalApi): authorize result action = CreateChild actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Fleet result = Ok(())
31912025-04-30T06:13:12.898ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31922025-04-30T06:13:12.898ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31932025-04-30T06:13:12.898ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31942025-04-30T06:13:12.910ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
31952025-04-30T06:13:12.910ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
31962025-04-30T06:13:12.910ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
31972025-04-30T06:13:12.910ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
31982025-04-30T06:13:12.911ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
31992025-04-30T06:13:12.911ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 4 reason = Signaled
32002025-04-30T06:13:12.911ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
32012025-04-30T06:13:12.911ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32022025-04-30T06:13:12.911ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
32032025-04-30T06:13:12.911ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32042025-04-30T06:13:12.912ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
32052025-04-30T06:13:12.921ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32062025-04-30T06:13:12.925ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32072025-04-30T06:13:12.926ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
32082025-04-30T06:13:12.926ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32092025-04-30T06:13:12.926ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
32102025-04-30T06:13:12.927ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32112025-04-30T06:13:12.931ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
32122025-04-30T06:13:12.931ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
32132025-04-30T06:13:12.931ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
32142025-04-30T06:13:12.931ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
32152025-04-30T06:13:12.935ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32162025-04-30T06:13:12.935ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
32172025-04-30T06:13:12.936ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32182025-04-30T06:13:12.945ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
32192025-04-30T06:13:12.945ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
32202025-04-30T06:13:12.946ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: fc e0 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56752 req_id = 9f219cca-90f4-462c-9c97-514f9fff4e6d
32212025-04-30T06:13:12.946ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64736,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56752 req_id = 9f219cca-90f4-462c-9c97-514f9fff4e6d
32222025-04-30T06:13:12.947ZDEBGomicron-dev (store): query key key = _mgs._tcp
32232025-04-30T06:13:12.947ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
32242025-04-30T06:13:12.948ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56752 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9f219cca-90f4-462c-9c97-514f9fff4e6d
32252025-04-30T06:13:12.948ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2457, tv_nsec: 747531689 } })
32262025-04-30T06:13:12.948ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
32272025-04-30T06:13:12.949ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 43 6c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e Cl..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49399 req_id = c7089024-3f9d-4585-86d6-ec1dbe0c4dbe
32282025-04-30T06:13:12.949ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17260,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49399 req_id = c7089024-3f9d-4585-86d6-ec1dbe0c4dbe
32292025-04-30T06:13:12.950ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
32302025-04-30T06:13:12.950ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49399 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c7089024-3f9d-4585-86d6-ec1dbe0c4dbe
32312025-04-30T06:13:12.985ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
32322025-04-30T06:13:13.037ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
32332025-04-30T06:13:13.037ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32342025-04-30T06:13:13.037ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32352025-04-30T06:13:13.037ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
32362025-04-30T06:13:13.037ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32372025-04-30T06:13:13.042ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32382025-04-30T06:13:13.047ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
32392025-04-30T06:13:13.047ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32402025-04-30T06:13:13.047ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
32412025-04-30T06:13:13.047ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32422025-04-30T06:13:13.055ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32432025-04-30T06:13:13.055ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
32442025-04-30T06:13:13.056ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32452025-04-30T06:13:13.089ZTRAComicron-dev (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32462025-04-30T06:13:13.089ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
32472025-04-30T06:13:13.091ZTRAComicron-dev (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
32482025-04-30T06:13:13.140ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
32492025-04-30T06:13:13.140ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
32502025-04-30T06:13:13.140ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32512025-04-30T06:13:13.140ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
32522025-04-30T06:13:13.141ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32532025-04-30T06:13:13.144ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
32542025-04-30T06:13:13.149ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
32552025-04-30T06:13:13.149ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32562025-04-30T06:13:13.149ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
32572025-04-30T06:13:13.150ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32582025-04-30T06:13:13.152ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
32592025-04-30T06:13:13.152ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
32602025-04-30T06:13:13.153ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
32612025-04-30T06:13:13.159ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
32622025-04-30T06:13:13.160ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: b3 09 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52714 req_id = 0ea4a802-9a20-4842-aa8b-1d49fdc7795d
32632025-04-30T06:13:13.160ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 45833,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52714 req_id = 0ea4a802-9a20-4842-aa8b-1d49fdc7795d
32642025-04-30T06:13:13.161ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
32652025-04-30T06:13:13.161ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45833, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52714 req_id = 0ea4a802-9a20-4842-aa8b-1d49fdc7795d
32662025-04-30T06:13:13.161ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
32672025-04-30T06:13:13.161ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
32682025-04-30T06:13:13.161ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
32692025-04-30T06:13:13.164ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3910c72-f333-48fc-bcfd-fb258cced8bd", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32702025-04-30T06:13:13.166ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
32712025-04-30T06:13:13.175ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c55918b-ee28-491a-9441-a019cf496499", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32722025-04-30T06:13:13.175ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
32732025-04-30T06:13:13.179ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21b166ed-af84-4c69-8ee4-5ecbb80275c3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32742025-04-30T06:13:13.179ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
32752025-04-30T06:13:13.187ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ae1ea46-8fd4-43d0-b93d-060f43c39565", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32762025-04-30T06:13:13.188ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
32772025-04-30T06:13:13.190ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9922327a-2af2-40e6-a2e4-2edbe4a21334", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32782025-04-30T06:13:13.191ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
32792025-04-30T06:13:13.193ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb2bd46f-3062-4ae8-ac23-ef1043bcdb51", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32802025-04-30T06:13:13.194ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
32812025-04-30T06:13:13.197ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3862fa1c-1973-4d29-81ec-fc32cac8c5c7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32822025-04-30T06:13:13.197ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
32832025-04-30T06:13:13.200ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79789702-78d5-4d77-8e13-ad4a1dcafeae", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32842025-04-30T06:13:13.200ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
32852025-04-30T06:13:13.200ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
32862025-04-30T06:13:13.204ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5c11666-776a-4bb2-8e42-62cb5bd51a39", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32872025-04-30T06:13:13.204ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
32882025-04-30T06:13:13.206ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e907fb8-20a1-48cc-8065-6a74cad8ee6d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32892025-04-30T06:13:13.206ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
32902025-04-30T06:13:13.208ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "95abb4fd-6c75-4f53-b4b2-4ebdba01422b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32912025-04-30T06:13:13.208ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
32922025-04-30T06:13:13.209ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "066b8c3c-e1dd-4c4f-a553-3c8bbe55c01f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32932025-04-30T06:13:13.210ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
32942025-04-30T06:13:13.211ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6fc4ce17-fe18-40d1-b0f0-0a3c9c684419", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32952025-04-30T06:13:13.212ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
32962025-04-30T06:13:13.215ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "704a7b5d-01fb-45c5-988d-a6a4532bd541", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32972025-04-30T06:13:13.215ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
32982025-04-30T06:13:13.217ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5d884e0-3e2b-4b96-848d-c23a99084a54", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
32992025-04-30T06:13:13.217ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
33002025-04-30T06:13:13.219ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd982bab-0d05-4847-a893-5860b8b8a461", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33012025-04-30T06:13:13.219ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
33022025-04-30T06:13:13.220ZDEBGomicron-dev (producer-registration-task): client request body = Some(Body) method = POST uri = http://[::1]:12221/metrics/producers
33032025-04-30T06:13:13.222ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb45b05f-5eba-44c9-a11a-ba8e3b10cd50", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33042025-04-30T06:13:13.222ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
33052025-04-30T06:13:13.225ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc589c12-1d7b-4339-afdf-44a05df7b5ac", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33062025-04-30T06:13:13.225ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
33072025-04-30T06:13:13.227ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "da61c4e7-bc20-4f3c-bd10-6c747a982ad6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33082025-04-30T06:13:13.227ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
33092025-04-30T06:13:13.227ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5e35bf9-850a-4c51-81b4-b0f4a9e792e3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33102025-04-30T06:13:13.228ZINFOomicron-dev (dropshot_internal): accepted connection local_addr = [::1]:12221 remote_addr = [::1]:35987
33112025-04-30T06:13:13.228ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
33122025-04-30T06:13:13.228ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = POST remote_addr = [::1]:35987 req_id = 51447213-e19b-45f7-a63f-fb2d8eaf2fcc uri = /metrics/producers
33132025-04-30T06:13:13.229ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:35987 req_id = 51447213-e19b-45f7-a63f-fb2d8eaf2fcc resource = Database uri = /metrics/producers
33142025-04-30T06:13:13.229ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:35987 req_id = 51447213-e19b-45f7-a63f-fb2d8eaf2fcc roles = RoleSet { roles: {} } uri = /metrics/producers
33152025-04-30T06:13:13.230ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:35987 req_id = 51447213-e19b-45f7-a63f-fb2d8eaf2fcc resource = Database result = Ok(()) uri = /metrics/producers
33162025-04-30T06:13:13.230ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a82f3b7b-378a-4936-bb98-126dccd171bd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33172025-04-30T06:13:13.231ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
33182025-04-30T06:13:13.236ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a71ff78-71b2-4606-bf3c-ac7b585c18e5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33192025-04-30T06:13:13.236ZINFOomicron-dev (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
33202025-04-30T06:13:13.236ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
33212025-04-30T06:13:13.237ZINFOomicron-dev (dropshot_internal): request completed latency_us = 8785 local_addr = [::1]:12221 method = POST remote_addr = [::1]:35987 req_id = 51447213-e19b-45f7-a63f-fb2d8eaf2fcc response_code = 201 uri = /metrics/producers
33222025-04-30T06:13:13.237ZDEBGomicron-dev (producer-registration-task): client response result = Ok(Response { url: "http://[::1]:12221/metrics/producers", status: 201, headers: {"content-type": "application/json", "x-request-id": "51447213-e19b-45f7-a63f-fb2d8eaf2fcc", "content-length": "41", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33232025-04-30T06:13:13.238ZDEBGomicron-dev (producer-registration-task): successfully registered with Nexus, pausing until time to renew lease lease_duration = 600s wait_period = 150s
33242025-04-30T06:13:13.238ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e272f54-25d8-4d14-bca6-e41ba1e4f6d9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33252025-04-30T06:13:13.239ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
33262025-04-30T06:13:13.240ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "30080317-6351-4590-963e-03fa8908e402", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33272025-04-30T06:13:13.241ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
33282025-04-30T06:13:13.243ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbd0d726-5b7a-4ed7-b650-2a7115a980c8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33292025-04-30T06:13:13.243ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
33302025-04-30T06:13:13.245ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2d7dccd-2820-476e-ab85-0dee8c1245c3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33312025-04-30T06:13:13.245ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
33322025-04-30T06:13:13.247ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb684321-2170-4c94-aedf-2faef5091d1c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33332025-04-30T06:13:13.247ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
33342025-04-30T06:13:13.250ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "876605c7-91a8-457c-9758-2b94dca4ea5d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33352025-04-30T06:13:13.250ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
33362025-04-30T06:13:13.252ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7094dc19-d87b-4dc5-bed8-02648465e459", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33372025-04-30T06:13:13.252ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
33382025-04-30T06:13:13.252ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
33392025-04-30T06:13:13.252ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33402025-04-30T06:13:13.252ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33412025-04-30T06:13:13.252ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
33422025-04-30T06:13:13.253ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33432025-04-30T06:13:13.254ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c384f344-fc6a-4ded-9e4f-77347f95c1f5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33442025-04-30T06:13:13.255ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
33452025-04-30T06:13:13.255ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cd47066-fafd-485e-8f2b-86608171c843", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33462025-04-30T06:13:13.255ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
33472025-04-30T06:13:13.256ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33482025-04-30T06:13:13.260ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c38a5856-cf40-4ae2-a054-c55b2c7a91b9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33492025-04-30T06:13:13.261ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
33502025-04-30T06:13:13.262ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
33512025-04-30T06:13:13.262ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33522025-04-30T06:13:13.262ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
33532025-04-30T06:13:13.263ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33542025-04-30T06:13:13.263ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "476dd1b2-d77b-45ac-8d66-73995e71ffcf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33552025-04-30T06:13:13.263ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
33562025-04-30T06:13:13.266ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33572025-04-30T06:13:13.266ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
33582025-04-30T06:13:13.267ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
33592025-04-30T06:13:13.269ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "21020b3f-80d3-412c-a962-0d0fe2e1bb73", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33602025-04-30T06:13:13.270ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
33612025-04-30T06:13:13.272ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e11775ee-a2ac-44c4-80db-3128fb7486ed", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33622025-04-30T06:13:13.273ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
33632025-04-30T06:13:13.276ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a896aa26-f8bc-4f00-afce-b4e947e7d531", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33642025-04-30T06:13:13.276ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
33652025-04-30T06:13:13.277ZINFOomicron-dev (dropshot_internal): accepted connection local_addr = [::1]:12221 remote_addr = [::1]:56144
33662025-04-30T06:13:13.277ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = POST remote_addr = [::1]:56144 req_id = 14cc4585-073a-4539-a9b1-9c327d52a43a uri = /metrics/producers
33672025-04-30T06:13:13.278ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:56144 req_id = 14cc4585-073a-4539-a9b1-9c327d52a43a resource = Database uri = /metrics/producers
33682025-04-30T06:13:13.278ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:56144 req_id = 14cc4585-073a-4539-a9b1-9c327d52a43a roles = RoleSet { roles: {} } uri = /metrics/producers
33692025-04-30T06:13:13.281ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = POST remote_addr = [::1]:56144 req_id = 14cc4585-073a-4539-a9b1-9c327d52a43a resource = Database result = Ok(()) uri = /metrics/producers
33702025-04-30T06:13:13.284ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79c871fe-1d96-48ab-ad92-38c2df36481d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:12 GMT"} })
33712025-04-30T06:13:13.284ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
33722025-04-30T06:13:13.288ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3097af6-dcf2-4c5c-a52f-b0b850e1a73d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33732025-04-30T06:13:13.289ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
33742025-04-30T06:13:13.297ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3c00662-9e9e-4e7f-afc1-5536a10e847a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33752025-04-30T06:13:13.299ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
33762025-04-30T06:13:13.301ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d83df08f-5bcb-45cf-95d2-4435d2c008ed", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33772025-04-30T06:13:13.301ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
33782025-04-30T06:13:13.324ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5b24469-3d1c-4c85-b81c-1a5ae2535176", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33792025-04-30T06:13:13.324ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
33802025-04-30T06:13:13.332ZINFOomicron-dev (nexus): assigned collector to new producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
33812025-04-30T06:13:13.332ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
33822025-04-30T06:13:13.332ZINFOomicron-dev (dropshot_internal): request completed latency_us = 54861 local_addr = [::1]:12221 method = POST remote_addr = [::1]:56144 req_id = 14cc4585-073a-4539-a9b1-9c327d52a43a response_code = 201 uri = /metrics/producers
33832025-04-30T06:13:13.334ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fb978bf-7711-4875-8bd9-70e614e7b6cf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33842025-04-30T06:13:13.334ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
33852025-04-30T06:13:13.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cf9f6eb-8517-451b-9eda-4208e6c4e5f3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33862025-04-30T06:13:13.337ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
33872025-04-30T06:13:13.344ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "435674d8-bfc4-4c28-8112-6569ea2b2454", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:12 GMT"} })
33882025-04-30T06:13:13.345ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
33892025-04-30T06:13:13.360ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "892c6bbc-56d0-4dfd-b5a2-73611c670b89", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
33902025-04-30T06:13:13.361ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
33912025-04-30T06:13:13.362ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33922025-04-30T06:13:13.362ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
33932025-04-30T06:13:13.362ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
33942025-04-30T06:13:13.363ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
33952025-04-30T06:13:13.368ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
33962025-04-30T06:13:13.383ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
33972025-04-30T06:13:13.384ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
33982025-04-30T06:13:13.384ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
33992025-04-30T06:13:13.384ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
34002025-04-30T06:13:13.384ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34012025-04-30T06:13:13.410ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
34022025-04-30T06:13:13.410ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
34032025-04-30T06:13:13.410ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34042025-04-30T06:13:13.410ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34052025-04-30T06:13:13.411ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34062025-04-30T06:13:13.412ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34072025-04-30T06:13:13.428ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34082025-04-30T06:13:13.428ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34092025-04-30T06:13:13.435ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
34102025-04-30T06:13:13.436ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34112025-04-30T06:13:13.436ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
34122025-04-30T06:13:13.436ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34132025-04-30T06:13:13.440ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
34142025-04-30T06:13:13.440ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34152025-04-30T06:13:13.440ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34162025-04-30T06:13:13.440ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34172025-04-30T06:13:13.440ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
34182025-04-30T06:13:13.441ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34192025-04-30T06:13:13.441ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34202025-04-30T06:13:13.483ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
34212025-04-30T06:13:13.534ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
34222025-04-30T06:13:13.535ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34232025-04-30T06:13:13.535ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34242025-04-30T06:13:13.535ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
34252025-04-30T06:13:13.535ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34262025-04-30T06:13:13.561ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
34272025-04-30T06:13:13.561ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
34282025-04-30T06:13:13.562ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:44776 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
34292025-04-30T06:13:13.573ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:44776 req_id = 3391af41-a909-47a4-9a07-129b3a7379ea sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
34302025-04-30T06:13:13.574ZINFOomicron-dev (dropshot): request completed latency_us = 701 local_addr = [::1]:61881 method = GET remote_addr = [::1]:44776 req_id = 3391af41-a909-47a4-9a07-129b3a7379ea response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
34312025-04-30T06:13:13.574ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34322025-04-30T06:13:13.580ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
34332025-04-30T06:13:13.580ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34342025-04-30T06:13:13.580ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
34352025-04-30T06:13:13.580ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3391af41-a909-47a4-9a07-129b3a7379ea", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
34362025-04-30T06:13:13.580ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34372025-04-30T06:13:13.584ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34382025-04-30T06:13:13.584ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
34392025-04-30T06:13:13.585ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34402025-04-30T06:13:13.586ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
34412025-04-30T06:13:13.587ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
34422025-04-30T06:13:13.587ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
34432025-04-30T06:13:13.589ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
34442025-04-30T06:13:13.589ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34452025-04-30T06:13:13.591ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34462025-04-30T06:13:13.591ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34472025-04-30T06:13:13.592ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34482025-04-30T06:13:13.624ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
34492025-04-30T06:13:13.658ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34502025-04-30T06:13:13.667ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
34512025-04-30T06:13:13.668ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34522025-04-30T06:13:13.669ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34532025-04-30T06:13:13.670ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34542025-04-30T06:13:13.674ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
34552025-04-30T06:13:13.674ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34562025-04-30T06:13:13.674ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34572025-04-30T06:13:13.675ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
34582025-04-30T06:13:13.675ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34592025-04-30T06:13:13.684ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34602025-04-30T06:13:13.690ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
34612025-04-30T06:13:13.690ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34622025-04-30T06:13:13.690ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
34632025-04-30T06:13:13.690ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34642025-04-30T06:13:13.830ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
34652025-04-30T06:13:13.831ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
34662025-04-30T06:13:13.831ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 920.053527ms iteration = 4 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
34672025-04-30T06:13:13.831ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 5 reason = Signaled
34682025-04-30T06:13:13.831ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
34692025-04-30T06:13:13.831ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
34702025-04-30T06:13:13.831ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34712025-04-30T06:13:13.832ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34722025-04-30T06:13:13.834ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34732025-04-30T06:13:13.835ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
34742025-04-30T06:13:13.835ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
34752025-04-30T06:13:13.836ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
34762025-04-30T06:13:13.841ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
34772025-04-30T06:13:13.845ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
34782025-04-30T06:13:13.845ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
34792025-04-30T06:13:13.845ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
34802025-04-30T06:13:13.846ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
34812025-04-30T06:13:13.864ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
34822025-04-30T06:13:13.864ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
34832025-04-30T06:13:13.870ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 1a 05 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51699 req_id = 8dac486a-58e9-4608-bc7e-2867cf1bbe88
34842025-04-30T06:13:13.870ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6661,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51699 req_id = 8dac486a-58e9-4608-bc7e-2867cf1bbe88
34852025-04-30T06:13:13.870ZDEBGomicron-dev (store): query key key = _mgs._tcp
34862025-04-30T06:13:13.870ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
34872025-04-30T06:13:13.871ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51699 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8dac486a-58e9-4608-bc7e-2867cf1bbe88
34882025-04-30T06:13:13.871ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2458, tv_nsec: 670509722 } })
34892025-04-30T06:13:13.871ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
34902025-04-30T06:13:13.872ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a6 75 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .u..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52587 req_id = 7d785370-e313-42fc-91c7-41d4fc68d99c
34912025-04-30T06:13:13.872ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42613,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52587 req_id = 7d785370-e313-42fc-91c7-41d4fc68d99c
34922025-04-30T06:13:13.872ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
34932025-04-30T06:13:13.872ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52587 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7d785370-e313-42fc-91c7-41d4fc68d99c
34942025-04-30T06:13:13.907ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
34952025-04-30T06:13:13.952ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
34962025-04-30T06:13:13.952ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 91 16 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64503 req_id = 453f4da6-cb33-412d-96e8-dcca6ab6f3ef
34972025-04-30T06:13:13.953ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37142,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64503 req_id = 453f4da6-cb33-412d-96e8-dcca6ab6f3ef
34982025-04-30T06:13:13.953ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
34992025-04-30T06:13:13.953ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 37142, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64503 req_id = 453f4da6-cb33-412d-96e8-dcca6ab6f3ef
35002025-04-30T06:13:13.953ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
35012025-04-30T06:13:13.953ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
35022025-04-30T06:13:13.953ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
35032025-04-30T06:13:13.956ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "adc184ec-49b5-4e5d-8d08-3f8b9187c08f", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35042025-04-30T06:13:13.956ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
35052025-04-30T06:13:13.958ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
35062025-04-30T06:13:13.958ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35072025-04-30T06:13:13.958ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35082025-04-30T06:13:13.958ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
35092025-04-30T06:13:13.959ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35102025-04-30T06:13:13.960ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e880c9be-603a-40b2-9e12-ad3e5546b30a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35112025-04-30T06:13:13.960ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
35122025-04-30T06:13:13.963ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "972008ea-8c8a-4c6e-89da-5901746b477f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35132025-04-30T06:13:13.963ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
35142025-04-30T06:13:13.963ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
35152025-04-30T06:13:13.966ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68a75451-3517-4539-9c99-833acd077f8d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35162025-04-30T06:13:13.966ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
35172025-04-30T06:13:13.968ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4494d1e7-c824-4c85-8513-b975e891aa1a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35182025-04-30T06:13:13.968ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
35192025-04-30T06:13:13.968ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
35202025-04-30T06:13:13.968ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35212025-04-30T06:13:13.969ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
35222025-04-30T06:13:13.969ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35232025-04-30T06:13:13.971ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d655142-e11b-42b8-8e8c-5f34f3163d2f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35242025-04-30T06:13:13.971ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
35252025-04-30T06:13:13.971ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
35262025-04-30T06:13:13.971ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
35272025-04-30T06:13:13.971ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
35282025-04-30T06:13:13.973ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f28434d-a8fd-408a-830e-8f0f1de7334b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35292025-04-30T06:13:13.974ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
35302025-04-30T06:13:13.975ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "de7dcbac-5afd-474a-bc57-84f1b2e63809", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35312025-04-30T06:13:13.976ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
35322025-04-30T06:13:13.976ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2b7c6f6-facd-4b03-94cc-a05dd5086a39", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35332025-04-30T06:13:13.977ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
35342025-04-30T06:13:13.978ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a050bc6-8caa-411e-8fd0-2e46166bda97", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35352025-04-30T06:13:13.978ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
35362025-04-30T06:13:13.980ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "82dbb615-af28-4dac-a1f9-4707855f19e8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35372025-04-30T06:13:13.980ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
35382025-04-30T06:13:13.981ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "76fe5cb7-cdfe-4093-9f4b-24c3bb63642e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35392025-04-30T06:13:13.981ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
35402025-04-30T06:13:13.982ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f022356-6f0e-4003-a818-3d320501f541", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35412025-04-30T06:13:13.983ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
35422025-04-30T06:13:13.987ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3089725-27ef-4333-bd13-1c146a7c1a73", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35432025-04-30T06:13:13.988ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
35442025-04-30T06:13:13.990ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bd2d466-88e9-4d8d-932e-83075ce76438", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35452025-04-30T06:13:13.991ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
35462025-04-30T06:13:13.994ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "344f0be8-8d34-48f7-8e89-4f9156b5d3aa", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35472025-04-30T06:13:13.994ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
35482025-04-30T06:13:13.995ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "615dd346-6ea9-4567-8047-b88a4766ac8c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35492025-04-30T06:13:13.995ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
35502025-04-30T06:13:13.997ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "084e932a-1a79-487b-9695-67ee2d9665fa", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35512025-04-30T06:13:13.997ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
35522025-04-30T06:13:13.999ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfdf6b49-de1b-49c9-8893-82f26675c0b3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35532025-04-30T06:13:13.999ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
35542025-04-30T06:13:14.001ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "192c2f62-ad31-4a52-8f32-a4298e359319", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35552025-04-30T06:13:14.002ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
35562025-04-30T06:13:14.003ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0706bd24-9b4b-4d2a-a4d0-bd1671b899de", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35572025-04-30T06:13:14.003ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
35582025-04-30T06:13:14.004ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
35592025-04-30T06:13:14.006ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fba905a6-1b09-4d5d-b151-a640d6d04dc5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35602025-04-30T06:13:14.006ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
35612025-04-30T06:13:14.007ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "06a24984-eb7f-4529-a03c-20dacb5b42ae", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35622025-04-30T06:13:14.007ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
35632025-04-30T06:13:14.009ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "86da9d4e-b7e5-4536-8062-0bffe59d6d5e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35642025-04-30T06:13:14.009ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
35652025-04-30T06:13:14.011ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f25d7c6-2254-48a2-8a62-b77ea7474929", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35662025-04-30T06:13:14.011ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
35672025-04-30T06:13:14.013ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b1bcca0-fdfa-4d6e-a206-73cbfcc0ba39", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35682025-04-30T06:13:14.013ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
35692025-04-30T06:13:14.015ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bb018f3-2dd8-4958-922a-4d35a6515620", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35702025-04-30T06:13:14.016ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
35712025-04-30T06:13:14.017ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f70c708e-263d-4f8e-a0b7-939a2bc7c08b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35722025-04-30T06:13:14.017ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
35732025-04-30T06:13:14.019ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0873473c-4e60-4df7-92d0-603c60f22fd6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35742025-04-30T06:13:14.019ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
35752025-04-30T06:13:14.021ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18eb8389-de47-4bc1-9cb6-a3208f66cdab", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35762025-04-30T06:13:14.021ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
35772025-04-30T06:13:14.022ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44845b16-01af-4fdd-bc60-c50f605f9a0d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35782025-04-30T06:13:14.022ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
35792025-04-30T06:13:14.026ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "145d05ed-c761-4356-a5ba-67ecba56b798", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35802025-04-30T06:13:14.026ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
35812025-04-30T06:13:14.029ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26b65081-c750-44f3-aa05-d3ebd5610efa", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35822025-04-30T06:13:14.029ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
35832025-04-30T06:13:14.033ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "968cc5c4-fce4-47ad-8e25-054a57c0f749", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35842025-04-30T06:13:14.033ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
35852025-04-30T06:13:14.035ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb081332-7102-47e2-94cd-00511b3fb1ad", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35862025-04-30T06:13:14.035ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
35872025-04-30T06:13:14.038ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9db0125b-4be1-4db9-bcab-e23941da7bd8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35882025-04-30T06:13:14.039ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
35892025-04-30T06:13:14.042ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1dbd22d-5479-4998-877d-e99856a41d92", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35902025-04-30T06:13:14.042ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
35912025-04-30T06:13:14.046ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "45d5a83e-65d3-496a-8f7f-51f4a0b57f5b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35922025-04-30T06:13:14.046ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
35932025-04-30T06:13:14.048ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3127b87-e312-43e8-b416-ad3c074a70a6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35942025-04-30T06:13:14.048ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
35952025-04-30T06:13:14.052ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16e0eee5-26aa-440d-9bc0-d54ccdbbfa25", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
35962025-04-30T06:13:14.052ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
35972025-04-30T06:13:14.056ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
35982025-04-30T06:13:14.056ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
35992025-04-30T06:13:14.056ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36002025-04-30T06:13:14.056ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
36012025-04-30T06:13:14.057ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f351fe84-bed5-4098-a7de-52cf1a147034", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
36022025-04-30T06:13:14.057ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
36032025-04-30T06:13:14.058ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "453df0fd-914c-446a-8596-9a6ddbb115e1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
36042025-04-30T06:13:14.058ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
36052025-04-30T06:13:14.059ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36062025-04-30T06:13:14.061ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44a8af6c-e8db-4210-a34c-0745d471c086", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
36072025-04-30T06:13:14.063ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
36082025-04-30T06:13:14.063ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36092025-04-30T06:13:14.067ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6df93efa-b4e8-4d7c-bb93-8a784cdb28c2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
36102025-04-30T06:13:14.071ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
36112025-04-30T06:13:14.072ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
36122025-04-30T06:13:14.072ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36132025-04-30T06:13:14.072ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
36142025-04-30T06:13:14.073ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36152025-04-30T06:13:14.075ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d69261e-e978-47c0-88b1-0bf0cc55ea9e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
36162025-04-30T06:13:14.076ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
36172025-04-30T06:13:14.076ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36182025-04-30T06:13:14.076ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36192025-04-30T06:13:14.076ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36202025-04-30T06:13:14.076ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36212025-04-30T06:13:14.078ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36222025-04-30T06:13:14.078ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
36232025-04-30T06:13:14.079ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36242025-04-30T06:13:14.080ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36252025-04-30T06:13:14.085ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
36262025-04-30T06:13:14.085ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
36272025-04-30T06:13:14.085ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36282025-04-30T06:13:14.085ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36292025-04-30T06:13:14.085ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36302025-04-30T06:13:14.086ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36312025-04-30T06:13:14.089ZTRAComicron-dev (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
36322025-04-30T06:13:14.090ZTRAComicron-dev (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
36332025-04-30T06:13:14.104ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36342025-04-30T06:13:14.105ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
36352025-04-30T06:13:14.105ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36362025-04-30T06:13:14.105ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36372025-04-30T06:13:14.105ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36382025-04-30T06:13:14.133ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
36392025-04-30T06:13:14.146ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
36402025-04-30T06:13:14.146ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
36412025-04-30T06:13:14.147ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:47502 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
36422025-04-30T06:13:14.147ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:47502 req_id = bed07564-033d-46f6-a88f-f87af98ec152 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
36432025-04-30T06:13:14.148ZINFOomicron-dev (dropshot): request completed latency_us = 1240 local_addr = [::1]:61881 method = GET remote_addr = [::1]:47502 req_id = bed07564-033d-46f6-a88f-f87af98ec152 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
36442025-04-30T06:13:14.149ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bed07564-033d-46f6-a88f-f87af98ec152", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:13 GMT"} })
36452025-04-30T06:13:14.152ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
36462025-04-30T06:13:14.152ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
36472025-04-30T06:13:14.152ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
36482025-04-30T06:13:14.154ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
36492025-04-30T06:13:14.154ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36502025-04-30T06:13:14.154ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36512025-04-30T06:13:14.154ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36522025-04-30T06:13:14.154ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36532025-04-30T06:13:14.159ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36542025-04-30T06:13:14.165ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
36552025-04-30T06:13:14.165ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36562025-04-30T06:13:14.165ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36572025-04-30T06:13:14.166ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36582025-04-30T06:13:14.185ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
36592025-04-30T06:13:14.185ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36602025-04-30T06:13:14.185ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36612025-04-30T06:13:14.185ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
36622025-04-30T06:13:14.185ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36632025-04-30T06:13:14.188ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36642025-04-30T06:13:14.198ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
36652025-04-30T06:13:14.198ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36662025-04-30T06:13:14.198ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
36672025-04-30T06:13:14.200ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36682025-04-30T06:13:14.205ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
36692025-04-30T06:13:14.205ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
36702025-04-30T06:13:14.205ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
36712025-04-30T06:13:14.205ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
36722025-04-30T06:13:14.205ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 373.828559ms iteration = 5 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
36732025-04-30T06:13:14.206ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 6 reason = Signaled
36742025-04-30T06:13:14.206ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
36752025-04-30T06:13:14.206ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
36762025-04-30T06:13:14.206ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
36772025-04-30T06:13:14.206ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36782025-04-30T06:13:14.207ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36792025-04-30T06:13:14.208ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36802025-04-30T06:13:14.221ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
36812025-04-30T06:13:14.228ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
36822025-04-30T06:13:14.228ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
36832025-04-30T06:13:14.229ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
36842025-04-30T06:13:14.229ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
36852025-04-30T06:13:14.241ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
36862025-04-30T06:13:14.244ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
36872025-04-30T06:13:14.247ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d1 f7 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53300 req_id = 1950e952-87a4-4e65-ac5f-2cdc5c143355
36882025-04-30T06:13:14.247ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53751,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53300 req_id = 1950e952-87a4-4e65-ac5f-2cdc5c143355
36892025-04-30T06:13:14.247ZDEBGomicron-dev (store): query key key = _mgs._tcp
36902025-04-30T06:13:14.248ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
36912025-04-30T06:13:14.248ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53300 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1950e952-87a4-4e65-ac5f-2cdc5c143355
36922025-04-30T06:13:14.248ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2459, tv_nsec: 47696472 } })
36932025-04-30T06:13:14.248ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
36942025-04-30T06:13:14.249ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 09 b9 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62609 req_id = c58abb61-c4b4-4253-8084-3c29562650db
36952025-04-30T06:13:14.250ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2489,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62609 req_id = c58abb61-c4b4-4253-8084-3c29562650db
36962025-04-30T06:13:14.250ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
36972025-04-30T06:13:14.250ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62609 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c58abb61-c4b4-4253-8084-3c29562650db
36982025-04-30T06:13:14.264ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
36992025-04-30T06:13:14.313ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
37002025-04-30T06:13:14.314ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 4f 59 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c OY..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52617 req_id = b659ba50-0b72-4485-acce-4e25490a8613
37012025-04-30T06:13:14.314ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20313,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52617 req_id = b659ba50-0b72-4485-acce-4e25490a8613
37022025-04-30T06:13:14.315ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
37032025-04-30T06:13:14.315ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 20313, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52617 req_id = b659ba50-0b72-4485-acce-4e25490a8613
37042025-04-30T06:13:14.315ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
37052025-04-30T06:13:14.315ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
37062025-04-30T06:13:14.315ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
37072025-04-30T06:13:14.315ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
37082025-04-30T06:13:14.316ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37092025-04-30T06:13:14.316ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
37102025-04-30T06:13:14.316ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
37112025-04-30T06:13:14.317ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37122025-04-30T06:13:14.320ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "57739479-2b9d-463b-aa3a-1def0d7abdda", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37132025-04-30T06:13:14.320ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
37142025-04-30T06:13:14.321ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37152025-04-30T06:13:14.329ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
37162025-04-30T06:13:14.330ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37172025-04-30T06:13:14.330ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
37182025-04-30T06:13:14.330ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37192025-04-30T06:13:14.334ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b7a0804-e067-4fae-8bae-e5bd198c9cbd", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37202025-04-30T06:13:14.334ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37212025-04-30T06:13:14.335ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
37222025-04-30T06:13:14.335ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
37232025-04-30T06:13:14.337ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37242025-04-30T06:13:14.340ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbd6b5a9-37f8-4d6a-a34c-3027ef2c1500", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37252025-04-30T06:13:14.340ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
37262025-04-30T06:13:14.343ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "15d8c56e-2e2b-4d17-923b-92bfdf98f674", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37272025-04-30T06:13:14.343ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
37282025-04-30T06:13:14.346ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fadd58ae-d0df-425f-aea8-36924f3aabb6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37292025-04-30T06:13:14.347ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
37302025-04-30T06:13:14.350ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9efb39ac-5c6c-4768-98fc-53ed069df374", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37312025-04-30T06:13:14.350ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
37322025-04-30T06:13:14.353ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e196637-d006-4eb0-a759-1569123d3857", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37332025-04-30T06:13:14.353ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
37342025-04-30T06:13:14.355ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee888aa0-0a32-4804-82a5-c146428a4cf0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37352025-04-30T06:13:14.355ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
37362025-04-30T06:13:14.356ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff582e63-7870-42d7-9964-8c8e50389f28", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37372025-04-30T06:13:14.356ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
37382025-04-30T06:13:14.360ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "537cca96-1318-4294-a56e-ec3b04648221", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37392025-04-30T06:13:14.360ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
37402025-04-30T06:13:14.364ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19233fda-b0ff-4e55-af8e-7924766fe7d6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37412025-04-30T06:13:14.364ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
37422025-04-30T06:13:14.365ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26b30aa5-0a0b-4045-8c99-c3a982451df7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37432025-04-30T06:13:14.366ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
37442025-04-30T06:13:14.368ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e7138ee-501a-48ad-8fcb-a32989019f96", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37452025-04-30T06:13:14.368ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
37462025-04-30T06:13:14.373ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea24e8e3-cfdb-47ad-93f0-2739e3294931", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37472025-04-30T06:13:14.373ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
37482025-04-30T06:13:14.383ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c9cc02e-3ed3-4ed3-87f7-954acf2c4ce4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37492025-04-30T06:13:14.384ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
37502025-04-30T06:13:14.387ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
37512025-04-30T06:13:14.389ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c25d59e-0953-401e-a877-bbe44c622615", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37522025-04-30T06:13:14.389ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
37532025-04-30T06:13:14.393ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5211908e-3ec9-4e2d-a4b5-d8dacd8f311c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37542025-04-30T06:13:14.394ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
37552025-04-30T06:13:14.396ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "845dd889-c63c-421b-984e-51be857939f6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37562025-04-30T06:13:14.396ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
37572025-04-30T06:13:14.429ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bf6a232-5b5e-4b1f-876a-71ac02145043", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37582025-04-30T06:13:14.433ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
37592025-04-30T06:13:14.441ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8205e2a7-d53b-4a2f-aee1-c280804f8294", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37602025-04-30T06:13:14.441ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
37612025-04-30T06:13:14.441ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
37622025-04-30T06:13:14.442ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
37632025-04-30T06:13:14.442ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37642025-04-30T06:13:14.442ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
37652025-04-30T06:13:14.443ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37662025-04-30T06:13:14.449ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
37672025-04-30T06:13:14.452ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "379bcb68-bdfe-4066-ab1a-b2ef72e146e8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37682025-04-30T06:13:14.452ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
37692025-04-30T06:13:14.457ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
37702025-04-30T06:13:14.457ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37712025-04-30T06:13:14.457ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
37722025-04-30T06:13:14.457ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37732025-04-30T06:13:14.458ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3f58008-7e42-43dd-8ebd-72eb5c4c9a58", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37742025-04-30T06:13:14.458ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
37752025-04-30T06:13:14.460ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
37762025-04-30T06:13:14.460ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
37772025-04-30T06:13:14.461ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
37782025-04-30T06:13:14.464ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d274bbf-6fc0-4334-a656-f5ab81dd7ac6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37792025-04-30T06:13:14.465ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
37802025-04-30T06:13:14.473ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "893eacdf-3d00-473f-889d-92d12d174684", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37812025-04-30T06:13:14.474ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
37822025-04-30T06:13:14.477ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6c22ef3-2929-429d-8a78-482cba761f02", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37832025-04-30T06:13:14.477ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
37842025-04-30T06:13:14.482ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef2d8cde-8cd6-42f6-8ae6-de51946a6c35", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37852025-04-30T06:13:14.482ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
37862025-04-30T06:13:14.487ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "71f291e7-f742-4ad5-add6-b0db445b44ac", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37872025-04-30T06:13:14.487ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
37882025-04-30T06:13:14.491ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c53f7fa-3e59-4dee-ba76-169f66b5d6e2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37892025-04-30T06:13:14.491ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
37902025-04-30T06:13:14.496ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
37912025-04-30T06:13:14.505ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fcfb95f-d8a9-4799-93be-e2633893e131", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37922025-04-30T06:13:14.505ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
37932025-04-30T06:13:14.507ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d0fbda2-6e84-4f62-883c-84e240cd44b3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37942025-04-30T06:13:14.508ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
37952025-04-30T06:13:14.509ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "045d01c5-567a-4c5f-a66d-759f93e9e57e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37962025-04-30T06:13:14.509ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
37972025-04-30T06:13:14.510ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "401a14ea-ca0b-462a-ad8e-dadd0af56f1c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
37982025-04-30T06:13:14.510ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
37992025-04-30T06:13:14.514ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a5f8b53-b4a5-45e5-85b2-2cfec625ad12", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38002025-04-30T06:13:14.514ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
38012025-04-30T06:13:14.517ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa4bac32-58d7-4f3d-a59f-ffd6650668be", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38022025-04-30T06:13:14.517ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
38032025-04-30T06:13:14.519ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "192c1371-dcb0-4a67-b15d-e7cd5135b414", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38042025-04-30T06:13:14.519ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
38052025-04-30T06:13:14.527ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6fab2ad-f07e-4b56-9075-c99a42f2ccaf", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38062025-04-30T06:13:14.527ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
38072025-04-30T06:13:14.532ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "936dabda-5772-4a58-a41f-8cbd27867511", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38082025-04-30T06:13:14.533ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
38092025-04-30T06:13:14.536ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd0a4d4f-5d23-4c17-8b55-ca55e27d70dc", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38102025-04-30T06:13:14.538ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
38112025-04-30T06:13:14.542ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "214809b7-fec9-4fc5-ae0b-6bbaf7a86c7e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38122025-04-30T06:13:14.542ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
38132025-04-30T06:13:14.545ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eee2aaa8-81af-452d-9000-c0c0db589789", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38142025-04-30T06:13:14.545ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
38152025-04-30T06:13:14.547ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "56677d4d-bd23-4f22-974f-a4eaf1ec7955", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38162025-04-30T06:13:14.547ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
38172025-04-30T06:13:14.548ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
38182025-04-30T06:13:14.548ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38192025-04-30T06:13:14.548ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38202025-04-30T06:13:14.548ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
38212025-04-30T06:13:14.549ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38222025-04-30T06:13:14.549ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b808254-b9c7-4d97-bdab-46dc1c24f2eb", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38232025-04-30T06:13:14.550ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
38242025-04-30T06:13:14.553ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "93ea6a27-ad0d-4cf7-bda2-49687bfb2167", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38252025-04-30T06:13:14.553ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
38262025-04-30T06:13:14.553ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38272025-04-30T06:13:14.555ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f6afff7-41eb-47e8-a2f1-a0ec27b135d0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38282025-04-30T06:13:14.555ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
38292025-04-30T06:13:14.556ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2dad926e-33bc-4a92-b501-634f2854c7c5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38302025-04-30T06:13:14.556ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
38312025-04-30T06:13:14.556ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38322025-04-30T06:13:14.556ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38332025-04-30T06:13:14.556ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38342025-04-30T06:13:14.557ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38352025-04-30T06:13:14.561ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38362025-04-30T06:13:14.565ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
38372025-04-30T06:13:14.565ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
38382025-04-30T06:13:14.565ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38392025-04-30T06:13:14.565ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38402025-04-30T06:13:14.565ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38412025-04-30T06:13:14.566ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38422025-04-30T06:13:14.569ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
38432025-04-30T06:13:14.569ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38442025-04-30T06:13:14.569ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
38452025-04-30T06:13:14.570ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38462025-04-30T06:13:14.570ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38472025-04-30T06:13:14.579ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
38482025-04-30T06:13:14.580ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38492025-04-30T06:13:14.580ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38502025-04-30T06:13:14.580ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38512025-04-30T06:13:14.584ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38522025-04-30T06:13:14.584ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
38532025-04-30T06:13:14.585ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38542025-04-30T06:13:14.644ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
38552025-04-30T06:13:14.646ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
38562025-04-30T06:13:14.646ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
38572025-04-30T06:13:14.646ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:61903 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
38582025-04-30T06:13:14.648ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:61903 req_id = a8908fae-4050-49ea-98b3-173fede78a67 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
38592025-04-30T06:13:14.648ZINFOomicron-dev (dropshot): request completed latency_us = 550 local_addr = [::1]:61881 method = GET remote_addr = [::1]:61903 req_id = a8908fae-4050-49ea-98b3-173fede78a67 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
38602025-04-30T06:13:14.648ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8908fae-4050-49ea-98b3-173fede78a67", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38612025-04-30T06:13:14.649ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
38622025-04-30T06:13:14.649ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
38632025-04-30T06:13:14.649ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
38642025-04-30T06:13:14.652ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
38652025-04-30T06:13:14.652ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38662025-04-30T06:13:14.652ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38672025-04-30T06:13:14.652ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38682025-04-30T06:13:14.653ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38692025-04-30T06:13:14.657ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38702025-04-30T06:13:14.664ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
38712025-04-30T06:13:14.664ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
38722025-04-30T06:13:14.664ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
38732025-04-30T06:13:14.665ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
38742025-04-30T06:13:14.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
38752025-04-30T06:13:14.680ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
38762025-04-30T06:13:14.680ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 097c7ea6-d74b-4711-977a-69d14d861ae1 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
38772025-04-30T06:13:14.680ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 097c7ea6-d74b-4711-977a-69d14d861ae1 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
38782025-04-30T06:13:14.680ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 097c7ea6-d74b-4711-977a-69d14d861ae1 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
38792025-04-30T06:13:14.681ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 097c7ea6-d74b-4711-977a-69d14d861ae1 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
38802025-04-30T06:13:14.696ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
38812025-04-30T06:13:14.697ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
38822025-04-30T06:13:14.697ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38832025-04-30T06:13:14.698ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
38842025-04-30T06:13:14.698ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
38852025-04-30T06:13:14.698ZINFOomicron-dev (dropshot_internal): request completed latency_us = 18157 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 097c7ea6-d74b-4711-977a-69d14d861ae1 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
38862025-04-30T06:13:14.700ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "097c7ea6-d74b-4711-977a-69d14d861ae1", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38872025-04-30T06:13:14.700ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
38882025-04-30T06:13:14.700ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6d368e42-dce5-458f-8f32-4f78d9de2253 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
38892025-04-30T06:13:14.702ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
38902025-04-30T06:13:14.703ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6d368e42-dce5-458f-8f32-4f78d9de2253 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
38912025-04-30T06:13:14.703ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6d368e42-dce5-458f-8f32-4f78d9de2253 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
38922025-04-30T06:13:14.704ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6d368e42-dce5-458f-8f32-4f78d9de2253 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
38932025-04-30T06:13:14.716ZINFOomicron-dev (dropshot_internal): request completed latency_us = 15414 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6d368e42-dce5-458f-8f32-4f78d9de2253 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
38942025-04-30T06:13:14.716ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
38952025-04-30T06:13:14.716ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
38962025-04-30T06:13:14.716ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
38972025-04-30T06:13:14.716ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d368e42-dce5-458f-8f32-4f78d9de2253", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
38982025-04-30T06:13:14.717ZDEBGomicron-dev (oximeter-agent): registered new metric producer address = [::1]:37942 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
38992025-04-30T06:13:14.717ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
39002025-04-30T06:13:14.720ZDEBGomicron-dev (oximeter-agent): registered new metric producer address = [::1]:50290 collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
39012025-04-30T06:13:14.722ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
39022025-04-30T06:13:14.839ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
39032025-04-30T06:13:14.840ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
39042025-04-30T06:13:14.840ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 633.771328ms iteration = 6 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
39052025-04-30T06:13:14.840ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 7 reason = Signaled
39062025-04-30T06:13:14.840ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
39072025-04-30T06:13:14.840ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
39082025-04-30T06:13:14.840ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
39092025-04-30T06:13:14.840ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
39102025-04-30T06:13:14.841ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
39112025-04-30T06:13:14.843ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
39122025-04-30T06:13:14.843ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
39132025-04-30T06:13:14.844ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
39142025-04-30T06:13:14.848ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39152025-04-30T06:13:14.854ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
39162025-04-30T06:13:14.854ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
39172025-04-30T06:13:14.854ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
39182025-04-30T06:13:14.854ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
39192025-04-30T06:13:14.869ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
39202025-04-30T06:13:14.870ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
39212025-04-30T06:13:14.870ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b9 0f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55489 req_id = 110b107a-fcd4-4dc7-881a-f5ccd20e4db3
39222025-04-30T06:13:14.877ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47375,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55489 req_id = 110b107a-fcd4-4dc7-881a-f5ccd20e4db3
39232025-04-30T06:13:14.878ZDEBGomicron-dev (store): query key key = _mgs._tcp
39242025-04-30T06:13:14.878ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
39252025-04-30T06:13:14.878ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55489 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 110b107a-fcd4-4dc7-881a-f5ccd20e4db3
39262025-04-30T06:13:14.883ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2459, tv_nsec: 682622080 } })
39272025-04-30T06:13:14.885ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
39282025-04-30T06:13:14.885ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
39292025-04-30T06:13:14.886ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 38 60 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 8`..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56921 req_id = e90b2555-2a29-4efa-bcd3-61fa04466efe
39302025-04-30T06:13:14.886ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14432,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56921 req_id = e90b2555-2a29-4efa-bcd3-61fa04466efe
39312025-04-30T06:13:14.886ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
39322025-04-30T06:13:14.886ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56921 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e90b2555-2a29-4efa-bcd3-61fa04466efe
39332025-04-30T06:13:14.890ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
39342025-04-30T06:13:14.897ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
39352025-04-30T06:13:14.941ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
39362025-04-30T06:13:14.941ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
39372025-04-30T06:13:14.941ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
39382025-04-30T06:13:14.941ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
39392025-04-30T06:13:14.944ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
39402025-04-30T06:13:14.951ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
39412025-04-30T06:13:14.960ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
39422025-04-30T06:13:14.960ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
39432025-04-30T06:13:14.960ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
39442025-04-30T06:13:14.961ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
39452025-04-30T06:13:14.962ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
39462025-04-30T06:13:14.963ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: cd 5f 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ._..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62876 req_id = 8bfa47f2-213a-423c-9065-3db41daff5d2
39472025-04-30T06:13:14.963ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52575,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62876 req_id = 8bfa47f2-213a-423c-9065-3db41daff5d2
39482025-04-30T06:13:14.963ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
39492025-04-30T06:13:14.963ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52575, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62876 req_id = 8bfa47f2-213a-423c-9065-3db41daff5d2
39502025-04-30T06:13:14.964ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
39512025-04-30T06:13:14.964ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
39522025-04-30T06:13:14.964ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
39532025-04-30T06:13:14.964ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
39542025-04-30T06:13:14.964ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
39552025-04-30T06:13:14.965ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
39562025-04-30T06:13:14.969ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9428943a-1ff3-40bc-bbcc-f357b369e9a6", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39572025-04-30T06:13:14.969ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
39582025-04-30T06:13:14.974ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5d7dd56-090d-4bc3-8b34-42e66b2bba28", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39592025-04-30T06:13:14.974ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
39602025-04-30T06:13:14.977ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b144b0fe-fd53-48ac-ad32-13a1650231cc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39612025-04-30T06:13:14.977ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
39622025-04-30T06:13:14.983ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d920374-d48b-4351-8208-220be2b6e1f1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39632025-04-30T06:13:14.983ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
39642025-04-30T06:13:14.991ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "728c8e88-cf84-4049-b37a-f9e41be7214c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39652025-04-30T06:13:15.008ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
39662025-04-30T06:13:15.024ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
39672025-04-30T06:13:15.027ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "33a3204c-d0d1-4a72-b413-724bf5d00959", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39682025-04-30T06:13:15.028ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
39692025-04-30T06:13:15.034ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a9dbed1-fc0d-44f9-9e32-ce5ecb513ce1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39702025-04-30T06:13:15.034ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
39712025-04-30T06:13:15.037ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7b8b69a-95a7-43ee-99e0-45812935ddcc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39722025-04-30T06:13:15.037ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
39732025-04-30T06:13:15.040ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b16fdf36-b3d1-4427-bb80-0a94cc5dd31c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39742025-04-30T06:13:15.040ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
39752025-04-30T06:13:15.043ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bfb0916-ad87-4a83-b18f-96417d8cfcc1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39762025-04-30T06:13:15.043ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
39772025-04-30T06:13:15.053ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1711e5c-cc9c-4193-9a3b-9ca8f2b31de0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39782025-04-30T06:13:15.054ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
39792025-04-30T06:13:15.061ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c0cec7d-c46b-4137-80fc-ab87b273f3e6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39802025-04-30T06:13:15.062ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
39812025-04-30T06:13:15.064ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3925e428-5292-4980-b385-af24a2665826", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39822025-04-30T06:13:15.064ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
39832025-04-30T06:13:15.068ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f526040-bfd6-4807-86ab-a2260329f43a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39842025-04-30T06:13:15.068ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
39852025-04-30T06:13:15.070ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc156e73-ad44-48fa-93ae-553d32e9a641", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39862025-04-30T06:13:15.070ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
39872025-04-30T06:13:15.072ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b69f96e6-1444-444a-b56e-aef0aa1aa826", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39882025-04-30T06:13:15.073ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
39892025-04-30T06:13:15.075ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7da0c28f-54eb-4b4b-90a4-974c8abae8b9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39902025-04-30T06:13:15.075ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
39912025-04-30T06:13:15.078ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9ebfe0e-8a97-49b0-8b43-89fd78d6edf8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39922025-04-30T06:13:15.078ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
39932025-04-30T06:13:15.080ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3399460-9ed4-4cc2-a9a1-c4bcb10723f8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39942025-04-30T06:13:15.080ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
39952025-04-30T06:13:15.080ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "214b21f9-b2f9-4393-a651-be2d5dd100e6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39962025-04-30T06:13:15.081ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
39972025-04-30T06:13:15.082ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d34b14b-1367-4c1b-980c-afca9be1d30f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
39982025-04-30T06:13:15.082ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
39992025-04-30T06:13:15.083ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "80a7caba-6e6e-41c8-8c03-4e010af65f17", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40002025-04-30T06:13:15.083ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
40012025-04-30T06:13:15.084ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "95335649-93f4-4a7a-97db-95e5daa6e24e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40022025-04-30T06:13:15.084ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
40032025-04-30T06:13:15.085ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "625b3575-4924-4eb4-ab9e-5c4ad097b583", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40042025-04-30T06:13:15.085ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
40052025-04-30T06:13:15.087ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eec5c72b-7fe8-43af-a338-b8fbfce7df94", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40062025-04-30T06:13:15.087ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
40072025-04-30T06:13:15.089ZTRAComicron-dev (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
40082025-04-30T06:13:15.089ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "873c36e5-2df6-46c2-a5c8-ebdd615039ca", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40092025-04-30T06:13:15.089ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
40102025-04-30T06:13:15.090ZTRAComicron-dev (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
40112025-04-30T06:13:15.091ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c219d45c-b02a-49e3-bb8a-d2060c4cc83b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40122025-04-30T06:13:15.091ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
40132025-04-30T06:13:15.092ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
40142025-04-30T06:13:15.092ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40152025-04-30T06:13:15.092ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40162025-04-30T06:13:15.092ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
40172025-04-30T06:13:15.093ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40182025-04-30T06:13:15.096ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "010b35c1-4c25-414a-8b6c-e68344aeff5a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40192025-04-30T06:13:15.096ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
40202025-04-30T06:13:15.097ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0ced866-7e00-44c2-9c2b-9e8bc69a15e1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40212025-04-30T06:13:15.097ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
40222025-04-30T06:13:15.098ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40232025-04-30T06:13:15.099ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d83daca5-cbab-49fe-a675-f92914bd6bdd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40242025-04-30T06:13:15.100ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
40252025-04-30T06:13:15.102ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db3d2c77-08c3-47f4-89f1-ef0c59aae98a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40262025-04-30T06:13:15.103ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
40272025-04-30T06:13:15.103ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
40282025-04-30T06:13:15.103ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40292025-04-30T06:13:15.103ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
40302025-04-30T06:13:15.103ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40312025-04-30T06:13:15.104ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "486dd1aa-4a3d-46c6-b43a-21eee1507ed7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40322025-04-30T06:13:15.104ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
40332025-04-30T06:13:15.106ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40342025-04-30T06:13:15.106ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
40352025-04-30T06:13:15.107ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40362025-04-30T06:13:15.107ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73f329b0-9656-4941-907e-cf1cb6aab586", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40372025-04-30T06:13:15.108ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
40382025-04-30T06:13:15.111ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4eb3a42-ad83-4d02-9a4a-396300057fba", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40392025-04-30T06:13:15.111ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
40402025-04-30T06:13:15.113ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d2a0427-68ce-4bdd-b4db-2f20625b3fb7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40412025-04-30T06:13:15.113ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
40422025-04-30T06:13:15.115ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fb0068c-c4fa-4868-8e1d-ff9df0952d8e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40432025-04-30T06:13:15.116ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
40442025-04-30T06:13:15.119ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "604dcefd-667b-42aa-9e90-59b35c261482", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40452025-04-30T06:13:15.119ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
40462025-04-30T06:13:15.122ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cfc4f4f-eb76-4f9c-9e00-e75f8570fe52", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40472025-04-30T06:13:15.122ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
40482025-04-30T06:13:15.124ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d417bd6-72a3-4e76-94b7-2e42671efed8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40492025-04-30T06:13:15.125ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
40502025-04-30T06:13:15.127ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "02ceedbc-f352-430f-ade4-64933ab23a11", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40512025-04-30T06:13:15.127ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
40522025-04-30T06:13:15.129ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bf11a29-3d61-4276-a049-940349f00064", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40532025-04-30T06:13:15.129ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
40542025-04-30T06:13:15.129ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8bc850f-d38a-4521-a976-8ed7683145b3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40552025-04-30T06:13:15.130ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
40562025-04-30T06:13:15.131ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e2d0d2a-3a88-44fc-a10e-407b2ef61036", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40572025-04-30T06:13:15.131ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
40582025-04-30T06:13:15.133ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "79803cc4-a78d-4f2d-8d94-5f02e0b7592e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40592025-04-30T06:13:15.133ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
40602025-04-30T06:13:15.135ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e558ba1-18d6-4c97-8109-841c7ba80cff", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40612025-04-30T06:13:15.135ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
40622025-04-30T06:13:15.135ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40632025-04-30T06:13:15.135ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40642025-04-30T06:13:15.135ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40652025-04-30T06:13:15.137ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40662025-04-30T06:13:15.149ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40672025-04-30T06:13:15.150ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
40682025-04-30T06:13:15.167ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
40692025-04-30T06:13:15.167ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
40702025-04-30T06:13:15.167ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40712025-04-30T06:13:15.167ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40722025-04-30T06:13:15.167ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40732025-04-30T06:13:15.167ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40742025-04-30T06:13:15.172ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40752025-04-30T06:13:15.177ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
40762025-04-30T06:13:15.177ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40772025-04-30T06:13:15.177ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40782025-04-30T06:13:15.178ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40792025-04-30T06:13:15.202ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
40802025-04-30T06:13:15.202ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40812025-04-30T06:13:15.202ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
40822025-04-30T06:13:15.202ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
40832025-04-30T06:13:15.203ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
40842025-04-30T06:13:15.264ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
40852025-04-30T06:13:15.264ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
40862025-04-30T06:13:15.265ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:59159 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
40872025-04-30T06:13:15.267ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:59159 req_id = 3859b877-ee8d-4eab-8b05-e91248dd724e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
40882025-04-30T06:13:15.267ZINFOomicron-dev (dropshot): request completed latency_us = 604 local_addr = [::1]:61881 method = GET remote_addr = [::1]:59159 req_id = 3859b877-ee8d-4eab-8b05-e91248dd724e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
40892025-04-30T06:13:15.268ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3859b877-ee8d-4eab-8b05-e91248dd724e", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
40902025-04-30T06:13:15.270ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
40912025-04-30T06:13:15.270ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
40922025-04-30T06:13:15.270ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
40932025-04-30T06:13:15.270ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
40942025-04-30T06:13:15.277ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
40952025-04-30T06:13:15.277ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
40962025-04-30T06:13:15.277ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
40972025-04-30T06:13:15.278ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
40982025-04-30T06:13:15.278ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
40992025-04-30T06:13:15.278ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
41002025-04-30T06:13:15.278ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41012025-04-30T06:13:15.278ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
41022025-04-30T06:13:15.278ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41032025-04-30T06:13:15.279ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41042025-04-30T06:13:15.282ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41052025-04-30T06:13:15.282ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
41062025-04-30T06:13:15.283ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41072025-04-30T06:13:15.292ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
41082025-04-30T06:13:15.292ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
41092025-04-30T06:13:15.293ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
41102025-04-30T06:13:15.294ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
41112025-04-30T06:13:15.352ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
41122025-04-30T06:13:15.404ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
41132025-04-30T06:13:15.404ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
41142025-04-30T06:13:15.404ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41152025-04-30T06:13:15.404ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
41162025-04-30T06:13:15.405ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41172025-04-30T06:13:15.417ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41182025-04-30T06:13:15.426ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
41192025-04-30T06:13:15.427ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41202025-04-30T06:13:15.427ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
41212025-04-30T06:13:15.427ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41222025-04-30T06:13:15.487ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
41232025-04-30T06:13:15.488ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
41242025-04-30T06:13:15.488ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 648.169596ms iteration = 7 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
41252025-04-30T06:13:15.488ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 8 reason = Signaled
41262025-04-30T06:13:15.488ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
41272025-04-30T06:13:15.488ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
41282025-04-30T06:13:15.488ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
41292025-04-30T06:13:15.488ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
41302025-04-30T06:13:15.489ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
41312025-04-30T06:13:15.493ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41322025-04-30T06:13:15.493ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41332025-04-30T06:13:15.493ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
41342025-04-30T06:13:15.495ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41352025-04-30T06:13:15.500ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
41362025-04-30T06:13:15.500ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
41372025-04-30T06:13:15.500ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
41382025-04-30T06:13:15.500ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
41392025-04-30T06:13:15.507ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
41402025-04-30T06:13:15.507ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
41412025-04-30T06:13:15.508ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 6f 0f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 o............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61074 req_id = 80f2699f-e063-4661-97a8-24f15a5ac35a
41422025-04-30T06:13:15.508ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28431,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61074 req_id = 80f2699f-e063-4661-97a8-24f15a5ac35a
41432025-04-30T06:13:15.509ZDEBGomicron-dev (store): query key key = _mgs._tcp
41442025-04-30T06:13:15.509ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
41452025-04-30T06:13:15.509ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61074 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 80f2699f-e063-4661-97a8-24f15a5ac35a
41462025-04-30T06:13:15.509ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2460, tv_nsec: 308719999 } })
41472025-04-30T06:13:15.509ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
41482025-04-30T06:13:15.510ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e7 58 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .X..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52951 req_id = a535294d-422e-4735-9fd4-00e98c55e118
41492025-04-30T06:13:15.510ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59224,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52951 req_id = a535294d-422e-4735-9fd4-00e98c55e118
41502025-04-30T06:13:15.511ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
41512025-04-30T06:13:15.511ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52951 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a535294d-422e-4735-9fd4-00e98c55e118
41522025-04-30T06:13:15.532ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
41532025-04-30T06:13:15.573ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
41542025-04-30T06:13:15.574ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ac 9a 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58782 req_id = 216bc892-a20b-4862-82d0-d06f2a65ad9f
41552025-04-30T06:13:15.574ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44186,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58782 req_id = 216bc892-a20b-4862-82d0-d06f2a65ad9f
41562025-04-30T06:13:15.574ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
41572025-04-30T06:13:15.574ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44186, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58782 req_id = 216bc892-a20b-4862-82d0-d06f2a65ad9f
41582025-04-30T06:13:15.575ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
41592025-04-30T06:13:15.575ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
41602025-04-30T06:13:15.575ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
41612025-04-30T06:13:15.578ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "594ee42e-2f33-4486-aa1a-45d712777c70", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41622025-04-30T06:13:15.578ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
41632025-04-30T06:13:15.581ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "50bc5792-1d7e-42c6-b631-b99aa0f0961d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41642025-04-30T06:13:15.582ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
41652025-04-30T06:13:15.584ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
41662025-04-30T06:13:15.584ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
41672025-04-30T06:13:15.584ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41682025-04-30T06:13:15.584ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
41692025-04-30T06:13:15.586ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41702025-04-30T06:13:15.589ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9c43287-3c03-4a22-87be-48fe675ffee7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41712025-04-30T06:13:15.589ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
41722025-04-30T06:13:15.601ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "16e68891-cf6b-4c40-9c25-4abdeba19b67", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41732025-04-30T06:13:15.601ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
41742025-04-30T06:13:15.607ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
41752025-04-30T06:13:15.608ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e20e53de-d12a-430e-a7cf-4073f414fe4e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41762025-04-30T06:13:15.608ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
41772025-04-30T06:13:15.610ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
41782025-04-30T06:13:15.610ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41792025-04-30T06:13:15.610ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
41802025-04-30T06:13:15.611ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41812025-04-30T06:13:15.616ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66f349e9-297d-4b83-a1ff-c8c127a19e77", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
41822025-04-30T06:13:15.616ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
41832025-04-30T06:13:15.616ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
41842025-04-30T06:13:15.616ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
41852025-04-30T06:13:15.616ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
41862025-04-30T06:13:15.618ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5c83dbb-47ad-4653-9c20-6c91fe26665a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:14 GMT"} })
41872025-04-30T06:13:15.618ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
41882025-04-30T06:13:15.625ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c9e67ae-ad0f-48dd-b622-0a21f5a09046", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41892025-04-30T06:13:15.625ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
41902025-04-30T06:13:15.628ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f383e2f-5c85-42ab-9562-5a15245a9b30", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41912025-04-30T06:13:15.628ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
41922025-04-30T06:13:15.631ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1777fb33-98e9-437f-b2c5-d2a315f327aa", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41932025-04-30T06:13:15.631ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
41942025-04-30T06:13:15.632ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37e6f77d-cde9-4be8-a80a-49b1c1cae531", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41952025-04-30T06:13:15.632ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
41962025-04-30T06:13:15.633ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f60c1ff2-ae80-4e59-8abf-5f2c991b9778", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41972025-04-30T06:13:15.634ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
41982025-04-30T06:13:15.635ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "580ced42-1f5e-4e3c-a7e0-3b94934d2869", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
41992025-04-30T06:13:15.635ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
42002025-04-30T06:13:15.637ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9bd16e34-1e02-4401-bbc8-4ae49a24d229", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42012025-04-30T06:13:15.637ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
42022025-04-30T06:13:15.639ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d517e2e8-1281-4d30-a0fd-da360c5fa7ad", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42032025-04-30T06:13:15.639ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
42042025-04-30T06:13:15.641ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac8be497-58a3-471a-b704-47339996d317", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42052025-04-30T06:13:15.641ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
42062025-04-30T06:13:15.643ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7072eb7d-def8-42df-acac-b51976a4f6d6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42072025-04-30T06:13:15.643ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
42082025-04-30T06:13:15.644ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "421337d8-ab4c-4258-9d5b-b9d20335f132", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42092025-04-30T06:13:15.644ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
42102025-04-30T06:13:15.646ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9747876-6e6a-4aa8-89b4-5b2d3496a5d2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42112025-04-30T06:13:15.646ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
42122025-04-30T06:13:15.647ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af30a12e-3b94-4f54-b04a-8d6be53a3ab8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42132025-04-30T06:13:15.647ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
42142025-04-30T06:13:15.648ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
42152025-04-30T06:13:15.648ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66028c3f-161b-4800-9d30-b95a26825ef3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42162025-04-30T06:13:15.649ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
42172025-04-30T06:13:15.653ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f354d8a-c7a1-4aec-b74f-8356822e57bf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42182025-04-30T06:13:15.653ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
42192025-04-30T06:13:15.655ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "20ec7c5d-2466-45e0-8e5b-2aa00bf838cb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42202025-04-30T06:13:15.655ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
42212025-04-30T06:13:15.659ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "740c62d0-1559-40d1-85a7-9878c933ffda", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42222025-04-30T06:13:15.659ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
42232025-04-30T06:13:15.663ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "90067f77-ae49-4546-a925-a704d1ec4581", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42242025-04-30T06:13:15.663ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
42252025-04-30T06:13:15.667ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fec074c1-d8bb-4068-a92c-20a976fc25b1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42262025-04-30T06:13:15.667ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
42272025-04-30T06:13:15.672ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f82b6129-4717-4f17-a2e9-8bf9011629e9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42282025-04-30T06:13:15.673ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
42292025-04-30T06:13:15.675ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3556622-6ba0-4406-b1a8-3b7578eef7cc", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42302025-04-30T06:13:15.675ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
42312025-04-30T06:13:15.677ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "328ed474-05ce-4e75-9577-824f82073ecc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42322025-04-30T06:13:15.677ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
42332025-04-30T06:13:15.679ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3141c67-9240-48c7-9606-83448ba100a3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42342025-04-30T06:13:15.679ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
42352025-04-30T06:13:15.680ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1ef67e8-4a22-4df2-a439-3b11e04c54b5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42362025-04-30T06:13:15.680ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
42372025-04-30T06:13:15.683ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3a570b0-6192-48fe-88e4-3592b34e567f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42382025-04-30T06:13:15.683ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
42392025-04-30T06:13:15.684ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bcd7d49-d30f-44ae-b984-39f66f67c6c2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42402025-04-30T06:13:15.685ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
42412025-04-30T06:13:15.686ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5d8b4a1-e058-4bad-a5bb-440e35b12483", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42422025-04-30T06:13:15.687ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
42432025-04-30T06:13:15.688ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "28fd2c42-45df-4845-9980-529e99971755", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42442025-04-30T06:13:15.688ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
42452025-04-30T06:13:15.692ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "07794971-edb0-4cc8-8e72-cf8d17dc2b05", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42462025-04-30T06:13:15.692ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
42472025-04-30T06:13:15.697ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa07e6c8-f962-4d4c-81a5-2f241f3129e6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42482025-04-30T06:13:15.697ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
42492025-04-30T06:13:15.699ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
42502025-04-30T06:13:15.699ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42512025-04-30T06:13:15.699ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
42522025-04-30T06:13:15.699ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
42532025-04-30T06:13:15.700ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
42542025-04-30T06:13:15.701ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "286589e2-3300-4f44-89d7-df280bb3c5ba", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42552025-04-30T06:13:15.701ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
42562025-04-30T06:13:15.703ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42572025-04-30T06:13:15.703ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "91335f44-872a-4113-bff9-8cd65b3e7a3e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42582025-04-30T06:13:15.703ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
42592025-04-30T06:13:15.705ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea10b114-7691-4654-be84-86ed01df32f8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42602025-04-30T06:13:15.705ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
42612025-04-30T06:13:15.707ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d34df86-f0fb-4f18-a134-5a8c9bbe16c4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42622025-04-30T06:13:15.707ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
42632025-04-30T06:13:15.708ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ec049be-e55b-4b76-9af7-ad37bf837cb0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42642025-04-30T06:13:15.708ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
42652025-04-30T06:13:15.709ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "359a4b06-5d1b-4995-a91d-53a903d066c1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42662025-04-30T06:13:15.709ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
42672025-04-30T06:13:15.710ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b92d3d3-5db4-4ddd-b550-fb612d719a11", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42682025-04-30T06:13:15.710ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
42692025-04-30T06:13:15.711ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
42702025-04-30T06:13:15.711ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
42712025-04-30T06:13:15.711ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
42722025-04-30T06:13:15.711ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e100abb5-751e-4625-be21-aa89179c5ec7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
42732025-04-30T06:13:15.711ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
42742025-04-30T06:13:15.711ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42752025-04-30T06:13:15.711ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42762025-04-30T06:13:15.711ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42772025-04-30T06:13:15.711ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
42782025-04-30T06:13:15.712ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42792025-04-30T06:13:15.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42802025-04-30T06:13:15.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42812025-04-30T06:13:15.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42822025-04-30T06:13:15.722ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42832025-04-30T06:13:15.723ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42842025-04-30T06:13:15.723ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
42852025-04-30T06:13:15.731ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
42862025-04-30T06:13:15.731ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
42872025-04-30T06:13:15.732ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
42882025-04-30T06:13:15.737ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42892025-04-30T06:13:15.751ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
42902025-04-30T06:13:15.751ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
42912025-04-30T06:13:15.752ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
42922025-04-30T06:13:15.753ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42932025-04-30T06:13:15.755ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42942025-04-30T06:13:15.759ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
42952025-04-30T06:13:15.763ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
42962025-04-30T06:13:15.774ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
42972025-04-30T06:13:15.774ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
42982025-04-30T06:13:15.774ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
42992025-04-30T06:13:15.774ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
43002025-04-30T06:13:15.783ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
43012025-04-30T06:13:15.835ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
43022025-04-30T06:13:15.835ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
43032025-04-30T06:13:15.835ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43042025-04-30T06:13:15.835ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
43052025-04-30T06:13:15.836ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43062025-04-30T06:13:15.870ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
43072025-04-30T06:13:15.870ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
43082025-04-30T06:13:15.870ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:44944 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
43092025-04-30T06:13:15.871ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:44944 req_id = 8383a090-e6bb-40ec-a062-53d5e9781747 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
43102025-04-30T06:13:15.872ZINFOomicron-dev (dropshot): request completed latency_us = 1132 local_addr = [::1]:61881 method = GET remote_addr = [::1]:44944 req_id = 8383a090-e6bb-40ec-a062-53d5e9781747 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
43112025-04-30T06:13:15.873ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8383a090-e6bb-40ec-a062-53d5e9781747", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
43122025-04-30T06:13:15.875ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43132025-04-30T06:13:15.876ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
43142025-04-30T06:13:15.876ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
43152025-04-30T06:13:15.876ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
43162025-04-30T06:13:15.878ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
43172025-04-30T06:13:15.878ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
43182025-04-30T06:13:15.878ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
43192025-04-30T06:13:15.878ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43202025-04-30T06:13:15.878ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
43212025-04-30T06:13:15.883ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43222025-04-30T06:13:15.886ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
43232025-04-30T06:13:15.886ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43242025-04-30T06:13:15.887ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
43252025-04-30T06:13:15.887ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43262025-04-30T06:13:15.897ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
43272025-04-30T06:13:15.897ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
43282025-04-30T06:13:15.897ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43292025-04-30T06:13:15.898ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
43302025-04-30T06:13:15.900ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43312025-04-30T06:13:15.901ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
43322025-04-30T06:13:15.902ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43332025-04-30T06:13:15.938ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
43342025-04-30T06:13:15.938ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
43352025-04-30T06:13:15.938ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
43362025-04-30T06:13:15.938ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 450.262008ms iteration = 8 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
43372025-04-30T06:13:15.938ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 9 reason = Signaled
43382025-04-30T06:13:15.938ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
43392025-04-30T06:13:15.939ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
43402025-04-30T06:13:15.939ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
43412025-04-30T06:13:15.939ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43422025-04-30T06:13:15.941ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
43432025-04-30T06:13:15.944ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43442025-04-30T06:13:15.952ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
43452025-04-30T06:13:15.952ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
43462025-04-30T06:13:15.952ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
43472025-04-30T06:13:15.953ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
43482025-04-30T06:13:15.957ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
43492025-04-30T06:13:15.958ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
43502025-04-30T06:13:15.960ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 31 19 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 1............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60338 req_id = f843f82c-2ce0-42b8-8e1b-83e17c50eac1
43512025-04-30T06:13:15.960ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12569,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60338 req_id = f843f82c-2ce0-42b8-8e1b-83e17c50eac1
43522025-04-30T06:13:15.960ZDEBGomicron-dev (store): query key key = _mgs._tcp
43532025-04-30T06:13:15.961ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
43542025-04-30T06:13:15.961ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60338 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f843f82c-2ce0-42b8-8e1b-83e17c50eac1
43552025-04-30T06:13:15.961ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2460, tv_nsec: 760853766 } })
43562025-04-30T06:13:15.962ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
43572025-04-30T06:13:15.963ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 1e 77 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .w..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58341 req_id = 4bcaa2c7-2fe2-4d6a-970d-90b67b3b62fc
43582025-04-30T06:13:15.964ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7799,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58341 req_id = 4bcaa2c7-2fe2-4d6a-970d-90b67b3b62fc
43592025-04-30T06:13:15.965ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
43602025-04-30T06:13:15.966ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58341 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 4bcaa2c7-2fe2-4d6a-970d-90b67b3b62fc
43612025-04-30T06:13:16.041ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
43622025-04-30T06:13:16.042ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
43632025-04-30T06:13:16.042ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
43642025-04-30T06:13:16.042ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43652025-04-30T06:13:16.042ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 61 f1 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c a............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:55486 req_id = 9d2bee2d-a79a-42a6-bbce-cec70dc646b2
43662025-04-30T06:13:16.042ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
43672025-04-30T06:13:16.042ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25073,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55486 req_id = 9d2bee2d-a79a-42a6-bbce-cec70dc646b2
43682025-04-30T06:13:16.042ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
43692025-04-30T06:13:16.043ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 25073, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:55486 req_id = 9d2bee2d-a79a-42a6-bbce-cec70dc646b2
43702025-04-30T06:13:16.043ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43712025-04-30T06:13:16.043ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
43722025-04-30T06:13:16.043ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
43732025-04-30T06:13:16.043ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
43742025-04-30T06:13:16.046ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
43752025-04-30T06:13:16.046ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "12b9f1b4-3c95-4b20-a182-d417e1877086", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
43762025-04-30T06:13:16.047ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
43772025-04-30T06:13:16.049ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ad28787-47fb-4c2d-888e-d545bb13563a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
43782025-04-30T06:13:16.050ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
43792025-04-30T06:13:16.052ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "942d29be-bc7d-4047-8ba5-a51d60f7c5ef", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
43802025-04-30T06:13:16.052ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
43812025-04-30T06:13:16.052ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
43822025-04-30T06:13:16.053ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43832025-04-30T06:13:16.053ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
43842025-04-30T06:13:16.053ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43852025-04-30T06:13:16.057ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
43862025-04-30T06:13:16.057ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
43872025-04-30T06:13:16.057ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
43882025-04-30T06:13:16.058ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1fe5d63-84ab-42d1-b996-531cb5c288b3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
43892025-04-30T06:13:16.058ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
43902025-04-30T06:13:16.061ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b201ab2-33e8-4bf3-b200-2e9a25946424", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
43912025-04-30T06:13:16.061ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
43922025-04-30T06:13:16.066ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c801a778-c9f9-4356-b592-fe2b8a93dda6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
43932025-04-30T06:13:16.066ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
43942025-04-30T06:13:16.073ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cbf9b97-7c78-4885-91c6-5b685f61c320", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
43952025-04-30T06:13:16.073ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
43962025-04-30T06:13:16.087ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "906480c5-ca90-4520-a4b3-61a11643ec24", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
43972025-04-30T06:13:16.088ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
43982025-04-30T06:13:16.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
43992025-04-30T06:13:16.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
44002025-04-30T06:13:16.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44012025-04-30T06:13:16.091ZDEBGomicron-dev (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
44022025-04-30T06:13:16.091ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = SELECT * FROM oximeter.timeseries_schema FORMAT Native;
44032025-04-30T06:13:16.092ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f020d65d-cce0-4dd4-a38f-b64418e2e7f3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44042025-04-30T06:13:16.092ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
44052025-04-30T06:13:16.095ZTRAComicron-dev (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
44062025-04-30T06:13:16.095ZDEBGomicron-dev (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e schema = TimeseriesSchema { timeseries_name: TimeseriesName("integration_target:integration_metric"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "metric_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "target_name", field_type: String, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:13:16.091557737Z } timeseries_name = integration_target:integration_metric
44072025-04-30T06:13:16.096ZDEBGomicron-dev (clickhouse-client): inserting 1 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
44082025-04-30T06:13:16.096ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 6 n_rows = 1 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
44092025-04-30T06:13:16.098ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "283bae69-05be-4575-ab0d-e90e6d7acd3a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44102025-04-30T06:13:16.098ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
44112025-04-30T06:13:16.102ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4e77a55-4330-4232-8f57-ae503b2e12f7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44122025-04-30T06:13:16.102ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
44132025-04-30T06:13:16.102ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e63aea0e-faf2-4b3a-abe9-27ff98798ea0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44142025-04-30T06:13:16.102ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
44152025-04-30T06:13:16.106ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a36a3964-0393-4b7a-aaf9-6c2e164d0ece", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44162025-04-30T06:13:16.106ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
44172025-04-30T06:13:16.106ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
44182025-04-30T06:13:16.106ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea6ea609-5268-4f64-99ef-fb68c70cf47a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44192025-04-30T06:13:16.106ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
44202025-04-30T06:13:16.107ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
44212025-04-30T06:13:16.107ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
44222025-04-30T06:13:16.111ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
44232025-04-30T06:13:16.111ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
44242025-04-30T06:13:16.113ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0634d28-e7e9-443b-b11f-647364bd8fca", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44252025-04-30T06:13:16.113ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
44262025-04-30T06:13:16.118ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
44272025-04-30T06:13:16.122ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba511e77-ad12-4523-9988-c5f11d220210", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44282025-04-30T06:13:16.122ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
44292025-04-30T06:13:16.125ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa4b814b-d252-48d5-8a2e-f6158d539d54", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44302025-04-30T06:13:16.125ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
44312025-04-30T06:13:16.127ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1e70d37-1a9b-401f-9fbe-c0fe9e7c1f6a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44322025-04-30T06:13:16.128ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
44332025-04-30T06:13:16.130ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "12b640ba-136a-4bd3-b61e-9a8c2b1bdbad", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44342025-04-30T06:13:16.130ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
44352025-04-30T06:13:16.132ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2155375e-ed8c-40a3-bfce-fcb0485c1379", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44362025-04-30T06:13:16.133ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
44372025-04-30T06:13:16.137ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfeaac1a-8c17-4651-9162-8689f1dc9ba0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44382025-04-30T06:13:16.137ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
44392025-04-30T06:13:16.140ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "175833e5-d622-4015-a672-48847ccaa75f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44402025-04-30T06:13:16.140ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
44412025-04-30T06:13:16.142ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "116adcbb-7a4b-4fb5-a399-76d53bca0f4c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44422025-04-30T06:13:16.142ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
44432025-04-30T06:13:16.143ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e3399f2-434f-4e9a-8f17-3a7922126700", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44442025-04-30T06:13:16.144ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
44452025-04-30T06:13:16.146ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ee8a753-4d4f-4b93-adf5-31dfcb7a517f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44462025-04-30T06:13:16.146ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
44472025-04-30T06:13:16.149ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "36cef5cd-28d0-4eb0-aa64-8b6fddd963af", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44482025-04-30T06:13:16.149ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
44492025-04-30T06:13:16.155ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9694a5d5-3c11-4729-a33d-82f8a5ffccee", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44502025-04-30T06:13:16.155ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
44512025-04-30T06:13:16.158ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "007d9242-2398-4082-8c4f-0092455810f8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44522025-04-30T06:13:16.158ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
44532025-04-30T06:13:16.161ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "54761dc6-749b-410a-856c-6c6255bdeef5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44542025-04-30T06:13:16.162ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
44552025-04-30T06:13:16.164ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "94789b2a-ea78-45e9-8422-223cd2028ac2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44562025-04-30T06:13:16.164ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
44572025-04-30T06:13:16.165ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4ca498c-ea1c-4800-b180-e6ad43a9900f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44582025-04-30T06:13:16.165ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
44592025-04-30T06:13:16.167ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6640b9e3-25c6-46e4-9dd1-3af2c76973ee", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44602025-04-30T06:13:16.167ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
44612025-04-30T06:13:16.168ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6304f0ff-4b93-4f2d-97e6-aabfaa665ce6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44622025-04-30T06:13:16.169ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
44632025-04-30T06:13:16.170ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
44642025-04-30T06:13:16.170ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
44652025-04-30T06:13:16.170ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
44662025-04-30T06:13:16.170ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
44672025-04-30T06:13:16.170ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "863a3e53-4146-41d9-9632-3be29cb73fe0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44682025-04-30T06:13:16.170ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
44692025-04-30T06:13:16.170ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
44702025-04-30T06:13:16.173ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a549617-b1ba-4789-8fd1-1a2e44ff761a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44712025-04-30T06:13:16.173ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
44722025-04-30T06:13:16.175ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
44732025-04-30T06:13:16.176ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "23720f88-0b0c-46b4-8690-aa56f8472d56", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44742025-04-30T06:13:16.177ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
44752025-04-30T06:13:16.183ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
44762025-04-30T06:13:16.183ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
44772025-04-30T06:13:16.183ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
44782025-04-30T06:13:16.183ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
44792025-04-30T06:13:16.184ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "066ddf2f-e88c-421a-a445-44af9a598b3a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44802025-04-30T06:13:16.184ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
44812025-04-30T06:13:16.187ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
44822025-04-30T06:13:16.187ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
44832025-04-30T06:13:16.188ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4d3b4cc-14e9-412a-80ac-d8b9dfe30da9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44842025-04-30T06:13:16.188ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
44852025-04-30T06:13:16.188ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
44862025-04-30T06:13:16.192ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
44872025-04-30T06:13:16.195ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2dc14472-7e0d-454f-9e9d-771d5ec84f28", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44882025-04-30T06:13:16.195ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
44892025-04-30T06:13:16.198ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0630b25f-7462-4914-950d-359ffb40d461", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44902025-04-30T06:13:16.198ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
44912025-04-30T06:13:16.201ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "35472eb5-3434-48fe-8af8-b90d076a1ef5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44922025-04-30T06:13:16.201ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
44932025-04-30T06:13:16.204ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "04c10687-d542-4dbf-881a-4a6a11566392", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44942025-04-30T06:13:16.204ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
44952025-04-30T06:13:16.206ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "699e30d1-7bc3-4461-8a07-32290386b607", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44962025-04-30T06:13:16.206ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
44972025-04-30T06:13:16.207ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3c9e9ca-8777-4317-94bd-69d312ec8a89", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
44982025-04-30T06:13:16.207ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
44992025-04-30T06:13:16.209ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "717669a5-f83a-4d49-800f-916f7739f4dd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
45002025-04-30T06:13:16.209ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
45012025-04-30T06:13:16.209ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45022025-04-30T06:13:16.209ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
45032025-04-30T06:13:16.209ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45042025-04-30T06:13:16.210ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45052025-04-30T06:13:16.215ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45062025-04-30T06:13:16.218ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
45072025-04-30T06:13:16.222ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
45082025-04-30T06:13:16.222ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
45092025-04-30T06:13:16.222ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45102025-04-30T06:13:16.222ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
45112025-04-30T06:13:16.222ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45122025-04-30T06:13:16.223ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45132025-04-30T06:13:16.227ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45142025-04-30T06:13:16.231ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
45152025-04-30T06:13:16.231ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
45162025-04-30T06:13:16.231ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45172025-04-30T06:13:16.231ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45182025-04-30T06:13:16.269ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
45192025-04-30T06:13:16.270ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45202025-04-30T06:13:16.270ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45212025-04-30T06:13:16.270ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
45222025-04-30T06:13:16.270ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45232025-04-30T06:13:16.289ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
45242025-04-30T06:13:16.289ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
45252025-04-30T06:13:16.290ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:53857 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
45262025-04-30T06:13:16.290ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:53857 req_id = 8e67f76d-677b-4292-af83-66744c5b5c26 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
45272025-04-30T06:13:16.291ZINFOomicron-dev (dropshot): request completed latency_us = 590 local_addr = [::1]:61881 method = GET remote_addr = [::1]:53857 req_id = 8e67f76d-677b-4292-af83-66744c5b5c26 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
45282025-04-30T06:13:16.291ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e67f76d-677b-4292-af83-66744c5b5c26", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
45292025-04-30T06:13:16.292ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45302025-04-30T06:13:16.295ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
45312025-04-30T06:13:16.295ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
45322025-04-30T06:13:16.295ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
45332025-04-30T06:13:16.297ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
45342025-04-30T06:13:16.298ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45352025-04-30T06:13:16.298ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
45362025-04-30T06:13:16.298ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45372025-04-30T06:13:16.298ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
45382025-04-30T06:13:16.298ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45392025-04-30T06:13:16.298ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
45402025-04-30T06:13:16.298ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45412025-04-30T06:13:16.298ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45422025-04-30T06:13:16.302ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45432025-04-30T06:13:16.302ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
45442025-04-30T06:13:16.302ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45452025-04-30T06:13:16.303ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45462025-04-30T06:13:16.309ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
45472025-04-30T06:13:16.310ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
45482025-04-30T06:13:16.310ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45492025-04-30T06:13:16.310ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45502025-04-30T06:13:16.341ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
45512025-04-30T06:13:16.392ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
45522025-04-30T06:13:16.393ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45532025-04-30T06:13:16.393ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45542025-04-30T06:13:16.393ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
45552025-04-30T06:13:16.394ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45562025-04-30T06:13:16.397ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45572025-04-30T06:13:16.407ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
45582025-04-30T06:13:16.407ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45592025-04-30T06:13:16.407ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
45602025-04-30T06:13:16.408ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45612025-04-30T06:13:16.428ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45622025-04-30T06:13:16.428ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
45632025-04-30T06:13:16.428ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
45642025-04-30T06:13:16.429ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
45652025-04-30T06:13:16.429ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 490.315321ms iteration = 9 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
45662025-04-30T06:13:16.429ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 10 reason = Signaled
45672025-04-30T06:13:16.429ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45682025-04-30T06:13:16.429ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
45692025-04-30T06:13:16.429ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45702025-04-30T06:13:16.429ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
45712025-04-30T06:13:16.429ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45722025-04-30T06:13:16.432ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45732025-04-30T06:13:16.439ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45742025-04-30T06:13:16.451ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
45752025-04-30T06:13:16.451ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
45762025-04-30T06:13:16.451ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
45772025-04-30T06:13:16.452ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
45782025-04-30T06:13:16.457ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
45792025-04-30T06:13:16.457ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
45802025-04-30T06:13:16.458ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 9c 2a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .*..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60800 req_id = 70b3a7e4-bf8c-4dfb-ab47-ad3945dd1a52
45812025-04-30T06:13:16.458ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39978,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60800 req_id = 70b3a7e4-bf8c-4dfb-ab47-ad3945dd1a52
45822025-04-30T06:13:16.458ZDEBGomicron-dev (store): query key key = _mgs._tcp
45832025-04-30T06:13:16.458ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
45842025-04-30T06:13:16.458ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60800 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 70b3a7e4-bf8c-4dfb-ab47-ad3945dd1a52
45852025-04-30T06:13:16.459ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2461, tv_nsec: 258063824 } })
45862025-04-30T06:13:16.459ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
45872025-04-30T06:13:16.460ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e9 87 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54450 req_id = 52530d24-24a9-4a40-a690-b06e51169cf5
45882025-04-30T06:13:16.460ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59783,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54450 req_id = 52530d24-24a9-4a40-a690-b06e51169cf5
45892025-04-30T06:13:16.460ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
45902025-04-30T06:13:16.460ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54450 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 52530d24-24a9-4a40-a690-b06e51169cf5
45912025-04-30T06:13:16.467ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
45922025-04-30T06:13:16.519ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
45932025-04-30T06:13:16.519ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
45942025-04-30T06:13:16.519ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
45952025-04-30T06:13:16.519ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
45962025-04-30T06:13:16.519ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
45972025-04-30T06:13:16.522ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
45982025-04-30T06:13:16.527ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
45992025-04-30T06:13:16.527ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
46002025-04-30T06:13:16.528ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
46012025-04-30T06:13:16.528ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
46022025-04-30T06:13:16.531ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
46032025-04-30T06:13:16.531ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
46042025-04-30T06:13:16.531ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
46052025-04-30T06:13:16.533ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
46062025-04-30T06:13:16.533ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 59 f4 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c Y............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64463 req_id = 9472cea2-e062-4983-a533-5b579de6e22c
46072025-04-30T06:13:16.534ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23028,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64463 req_id = 9472cea2-e062-4983-a533-5b579de6e22c
46082025-04-30T06:13:16.534ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
46092025-04-30T06:13:16.534ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 23028, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64463 req_id = 9472cea2-e062-4983-a533-5b579de6e22c
46102025-04-30T06:13:16.534ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
46112025-04-30T06:13:16.534ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
46122025-04-30T06:13:16.534ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
46132025-04-30T06:13:16.537ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f70edd4-17c8-4670-9678-103441deb099", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46142025-04-30T06:13:16.538ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
46152025-04-30T06:13:16.539ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5c5500c-825c-4f1a-85e6-2424c1a5df4b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46162025-04-30T06:13:16.539ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
46172025-04-30T06:13:16.543ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dab79106-94a6-4271-9bf5-9447e9df7793", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46182025-04-30T06:13:16.543ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
46192025-04-30T06:13:16.545ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fdb09107-75ab-4428-a639-31c8bf0dd676", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46202025-04-30T06:13:16.545ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
46212025-04-30T06:13:16.547ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b0795af-e39d-4f4f-b33c-ecfdfd5b68db", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46222025-04-30T06:13:16.547ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
46232025-04-30T06:13:16.549ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d31dbbe-4081-43ed-aefd-823fb12c91e3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46242025-04-30T06:13:16.549ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
46252025-04-30T06:13:16.551ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d600cc0f-357b-40cb-a32b-86424692d8a7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46262025-04-30T06:13:16.551ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
46272025-04-30T06:13:16.552ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fc8b626-c98d-415c-b1c6-4d1e6178c94e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46282025-04-30T06:13:16.553ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
46292025-04-30T06:13:16.553ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "957f626f-a5f2-4f9d-8023-c9765c16db80", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46302025-04-30T06:13:16.554ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
46312025-04-30T06:13:16.555ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "67b12e38-c9ec-47f2-9864-da0f991212fc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46322025-04-30T06:13:16.555ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
46332025-04-30T06:13:16.557ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7454cb01-d19f-42ab-8e83-5db68185fc33", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46342025-04-30T06:13:16.557ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
46352025-04-30T06:13:16.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4556a4a7-4e54-4385-a7d6-d854a7f32126", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46362025-04-30T06:13:16.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
46372025-04-30T06:13:16.560ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "daf25db8-1bb3-4a62-b389-a84a73c6e808", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46382025-04-30T06:13:16.560ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
46392025-04-30T06:13:16.562ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9553b06-e433-4e01-ab46-e7058a36337e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46402025-04-30T06:13:16.563ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
46412025-04-30T06:13:16.566ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d47b187e-e7bf-4c5b-a075-787a00bf6a0f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46422025-04-30T06:13:16.566ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
46432025-04-30T06:13:16.568ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d961fd2-06b9-42d5-8f14-3156f4462fe3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46442025-04-30T06:13:16.568ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
46452025-04-30T06:13:16.571ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b43afe85-2da5-4831-8b28-c330a99301fa", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46462025-04-30T06:13:16.572ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
46472025-04-30T06:13:16.573ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
46482025-04-30T06:13:16.575ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1abe029c-f7f9-4aba-9379-7301f6155f4d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46492025-04-30T06:13:16.575ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
46502025-04-30T06:13:16.577ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f84f2dc8-5f6d-46ef-a266-5518e9faaa39", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46512025-04-30T06:13:16.577ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
46522025-04-30T06:13:16.578ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e78f0e5-2396-4bd1-9b44-c966cdaef397", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46532025-04-30T06:13:16.578ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
46542025-04-30T06:13:16.580ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c597866-006d-462f-bca5-1187365d8901", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46552025-04-30T06:13:16.580ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
46562025-04-30T06:13:16.582ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b07c791-098a-4a25-9737-b13a92debbe9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46572025-04-30T06:13:16.583ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
46582025-04-30T06:13:16.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbb6b63f-bf4a-4e64-8b48-15fee7b3194e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46592025-04-30T06:13:16.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
46602025-04-30T06:13:16.591ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0833e751-66f6-4ffa-b62b-809db66105e7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46612025-04-30T06:13:16.591ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
46622025-04-30T06:13:16.591ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bed513d-7908-4100-b22e-08263f9a61e0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46632025-04-30T06:13:16.591ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
46642025-04-30T06:13:16.594ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c8a0fbb-c9a3-480a-ab15-93d97869d978", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:15 GMT"} })
46652025-04-30T06:13:16.594ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
46662025-04-30T06:13:16.601ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "823aa88c-66db-4fbd-bcbc-6c43f445d4cd", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46672025-04-30T06:13:16.601ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
46682025-04-30T06:13:16.610ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "12075d4b-a425-4b5c-bd16-6c4a356d01c3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46692025-04-30T06:13:16.610ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
46702025-04-30T06:13:16.612ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4742fa30-37bb-403a-b531-06f9fffa86f2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46712025-04-30T06:13:16.612ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
46722025-04-30T06:13:16.614ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b53a12a-9c18-4b4a-8563-dfd31f64b347", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46732025-04-30T06:13:16.614ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
46742025-04-30T06:13:16.615ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e48c99e8-063e-4685-8b19-69943b403536", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46752025-04-30T06:13:16.615ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
46762025-04-30T06:13:16.617ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c87b91aa-8b82-4beb-a955-e25b00747b7d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46772025-04-30T06:13:16.617ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
46782025-04-30T06:13:16.621ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1a8890a-1a42-426a-86fc-8b78fe9b1d45", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46792025-04-30T06:13:16.621ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
46802025-04-30T06:13:16.624ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd0e722d-2f66-4be7-945f-13f1344f9202", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46812025-04-30T06:13:16.624ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
46822025-04-30T06:13:16.629ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
46832025-04-30T06:13:16.629ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
46842025-04-30T06:13:16.629ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
46852025-04-30T06:13:16.629ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
46862025-04-30T06:13:16.630ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
46872025-04-30T06:13:16.631ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6aeb7ed9-24e7-4e4d-b376-4a6001b1a38b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46882025-04-30T06:13:16.631ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
46892025-04-30T06:13:16.636ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e4c725e-1181-43ad-95e0-3e03d87577f9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46902025-04-30T06:13:16.636ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
46912025-04-30T06:13:16.637ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
46922025-04-30T06:13:16.639ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "22c082b2-8dcf-489d-bb4e-3a333a98ea05", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46932025-04-30T06:13:16.639ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
46942025-04-30T06:13:16.642ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc4a13bf-63c4-48b8-970f-a9afe42bd183", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
46952025-04-30T06:13:16.642ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
46962025-04-30T06:13:16.643ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
46972025-04-30T06:13:16.643ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
46982025-04-30T06:13:16.643ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
46992025-04-30T06:13:16.643ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
47002025-04-30T06:13:16.644ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e50045ef-55bf-492e-9890-03c2df56e237", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47012025-04-30T06:13:16.644ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
47022025-04-30T06:13:16.646ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0d1d313-96ec-40bc-8430-58e87a529322", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47032025-04-30T06:13:16.646ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
47042025-04-30T06:13:16.646ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
47052025-04-30T06:13:16.646ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
47062025-04-30T06:13:16.648ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
47072025-04-30T06:13:16.648ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72b572c5-5b2e-459f-a065-b700109669b1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47082025-04-30T06:13:16.648ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
47092025-04-30T06:13:16.649ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec18447d-d5ac-4301-9928-a574b82c9be6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47102025-04-30T06:13:16.649ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
47112025-04-30T06:13:16.650ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6833f36-63ad-4128-b94d-4d2c2714b928", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47122025-04-30T06:13:16.650ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
47132025-04-30T06:13:16.651ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9270c40a-4829-447e-9f84-26c9bc1eebd6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47142025-04-30T06:13:16.651ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
47152025-04-30T06:13:16.652ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca0e0c2c-77bd-4d8c-83f1-61489b81e7bf", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47162025-04-30T06:13:16.653ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
47172025-04-30T06:13:16.653ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47182025-04-30T06:13:16.653ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
47192025-04-30T06:13:16.653ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47202025-04-30T06:13:16.653ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
47212025-04-30T06:13:16.657ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47222025-04-30T06:13:16.666ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
47232025-04-30T06:13:16.666ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
47242025-04-30T06:13:16.666ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47252025-04-30T06:13:16.666ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
47262025-04-30T06:13:16.666ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47272025-04-30T06:13:16.667ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
47282025-04-30T06:13:16.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
47292025-04-30T06:13:16.680ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47302025-04-30T06:13:16.680ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47312025-04-30T06:13:16.680ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 98c43659-9fc2-4a97-9352-f8ea0d0b0ff7 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47322025-04-30T06:13:16.680ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 98c43659-9fc2-4a97-9352-f8ea0d0b0ff7 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47332025-04-30T06:13:16.680ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 98c43659-9fc2-4a97-9352-f8ea0d0b0ff7 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47342025-04-30T06:13:16.681ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 98c43659-9fc2-4a97-9352-f8ea0d0b0ff7 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47352025-04-30T06:13:16.685ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
47362025-04-30T06:13:16.686ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
47372025-04-30T06:13:16.686ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47382025-04-30T06:13:16.686ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
47392025-04-30T06:13:16.690ZINFOomicron-dev (dropshot_internal): request completed latency_us = 9926 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 98c43659-9fc2-4a97-9352-f8ea0d0b0ff7 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
47402025-04-30T06:13:16.690ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
47412025-04-30T06:13:16.690ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "98c43659-9fc2-4a97-9352-f8ea0d0b0ff7", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47422025-04-30T06:13:16.691ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
47432025-04-30T06:13:16.691ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 92123f33-878c-46f7-8cc1-eee5e90830f6 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
47442025-04-30T06:13:16.691ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 92123f33-878c-46f7-8cc1-eee5e90830f6 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
47452025-04-30T06:13:16.692ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 92123f33-878c-46f7-8cc1-eee5e90830f6 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
47462025-04-30T06:13:16.693ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 92123f33-878c-46f7-8cc1-eee5e90830f6 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
47472025-04-30T06:13:16.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47482025-04-30T06:13:16.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47492025-04-30T06:13:16.719ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47502025-04-30T06:13:16.720ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47512025-04-30T06:13:16.720ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47522025-04-30T06:13:16.720ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47532025-04-30T06:13:16.728ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
47542025-04-30T06:13:16.728ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
47552025-04-30T06:13:16.729ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:37300 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
47562025-04-30T06:13:16.730ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:37300 req_id = d05a80f1-0968-42f2-8076-e51b44c467e9 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
47572025-04-30T06:13:16.731ZINFOomicron-dev (dropshot): request completed latency_us = 752 local_addr = [::1]:61881 method = GET remote_addr = [::1]:37300 req_id = d05a80f1-0968-42f2-8076-e51b44c467e9 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
47582025-04-30T06:13:16.732ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d05a80f1-0968-42f2-8076-e51b44c467e9", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47592025-04-30T06:13:16.734ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
47602025-04-30T06:13:16.734ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
47612025-04-30T06:13:16.734ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
47622025-04-30T06:13:16.735ZINFOomicron-dev (dropshot_internal): request completed latency_us = 44094 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 92123f33-878c-46f7-8cc1-eee5e90830f6 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
47632025-04-30T06:13:16.736ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "92123f33-878c-46f7-8cc1-eee5e90830f6", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
47642025-04-30T06:13:16.736ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
47652025-04-30T06:13:16.736ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
47662025-04-30T06:13:16.736ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
47672025-04-30T06:13:16.736ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
47682025-04-30T06:13:16.736ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47692025-04-30T06:13:16.736ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
47702025-04-30T06:13:16.736ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47712025-04-30T06:13:16.737ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
47722025-04-30T06:13:16.741ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
47732025-04-30T06:13:16.741ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47742025-04-30T06:13:16.741ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47752025-04-30T06:13:16.742ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
47762025-04-30T06:13:16.742ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
47772025-04-30T06:13:16.744ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
47782025-04-30T06:13:16.749ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
47792025-04-30T06:13:16.749ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
47802025-04-30T06:13:16.749ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47812025-04-30T06:13:16.750ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
47822025-04-30T06:13:16.755ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
47832025-04-30T06:13:16.762ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
47842025-04-30T06:13:16.762ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
47852025-04-30T06:13:16.762ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
47862025-04-30T06:13:16.762ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
47872025-04-30T06:13:16.765ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
47882025-04-30T06:13:16.765ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
47892025-04-30T06:13:16.766ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
47902025-04-30T06:13:16.784ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
47912025-04-30T06:13:16.784ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
47922025-04-30T06:13:16.784ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 355.31899ms iteration = 10 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
47932025-04-30T06:13:16.784ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 11 reason = Signaled
47942025-04-30T06:13:16.784ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
47952025-04-30T06:13:16.785ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
47962025-04-30T06:13:16.785ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
47972025-04-30T06:13:16.785ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
47982025-04-30T06:13:16.786ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
47992025-04-30T06:13:16.789ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48002025-04-30T06:13:16.792ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
48012025-04-30T06:13:16.793ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
48022025-04-30T06:13:16.793ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
48032025-04-30T06:13:16.793ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
48042025-04-30T06:13:16.794ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
48052025-04-30T06:13:16.797ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
48062025-04-30T06:13:16.797ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
48072025-04-30T06:13:16.798ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 4c e9 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 L............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54302 req_id = c68decc8-f23d-402e-9f06-efa75423eeb4
48082025-04-30T06:13:16.798ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19689,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54302 req_id = c68decc8-f23d-402e-9f06-efa75423eeb4
48092025-04-30T06:13:16.799ZDEBGomicron-dev (store): query key key = _mgs._tcp
48102025-04-30T06:13:16.799ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
48112025-04-30T06:13:16.799ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54302 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c68decc8-f23d-402e-9f06-efa75423eeb4
48122025-04-30T06:13:16.799ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2461, tv_nsec: 598574329 } })
48132025-04-30T06:13:16.799ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
48142025-04-30T06:13:16.800ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 21 84 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e !...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52296 req_id = 34b45550-88c3-422e-a6d1-b19ab0e6f67a
48152025-04-30T06:13:16.800ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8580,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52296 req_id = 34b45550-88c3-422e-a6d1-b19ab0e6f67a
48162025-04-30T06:13:16.801ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
48172025-04-30T06:13:16.801ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52296 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 34b45550-88c3-422e-a6d1-b19ab0e6f67a
48182025-04-30T06:13:16.845ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
48192025-04-30T06:13:16.846ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
48202025-04-30T06:13:16.846ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
48212025-04-30T06:13:16.846ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
48222025-04-30T06:13:16.846ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
48232025-04-30T06:13:16.847ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 67 05 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c g............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62381 req_id = dbf3cee1-7290-4b59-ad66-3969fed8847d
48242025-04-30T06:13:16.847ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26373,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62381 req_id = dbf3cee1-7290-4b59-ad66-3969fed8847d
48252025-04-30T06:13:16.847ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
48262025-04-30T06:13:16.847ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
48272025-04-30T06:13:16.847ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 26373, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62381 req_id = dbf3cee1-7290-4b59-ad66-3969fed8847d
48282025-04-30T06:13:16.849ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
48292025-04-30T06:13:16.849ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
48302025-04-30T06:13:16.849ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
48312025-04-30T06:13:16.851ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
48322025-04-30T06:13:16.854ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2cc1a0e-2788-4144-b801-ca683a381928", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48332025-04-30T06:13:16.854ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
48342025-04-30T06:13:16.856ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d35eb1a-65ff-45a7-8789-2a28832a2ceb", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48352025-04-30T06:13:16.856ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
48362025-04-30T06:13:16.858ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
48372025-04-30T06:13:16.858ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
48382025-04-30T06:13:16.858ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
48392025-04-30T06:13:16.860ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
48402025-04-30T06:13:16.860ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6e253d6-8dee-44c4-9896-99a54eb3c227", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48412025-04-30T06:13:16.860ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
48422025-04-30T06:13:16.864ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
48432025-04-30T06:13:16.865ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
48442025-04-30T06:13:16.865ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
48452025-04-30T06:13:16.866ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "09771f1f-c202-4828-8838-6454f929a69a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48462025-04-30T06:13:16.866ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
48472025-04-30T06:13:16.871ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cc03382-dda0-482d-b56a-ae3547ef4459", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48482025-04-30T06:13:16.871ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
48492025-04-30T06:13:16.874ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2dec6f8-9314-452a-b8bb-2f909f3ade11", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48502025-04-30T06:13:16.874ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
48512025-04-30T06:13:16.879ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16fc90c6-3eed-4862-9104-33ada784eb70", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48522025-04-30T06:13:16.879ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
48532025-04-30T06:13:16.882ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a3a9ed3-0fa6-4290-abe5-3e9f1be4c899", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48542025-04-30T06:13:16.882ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
48552025-04-30T06:13:16.883ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c05e68bc-b691-48fa-bb5c-2241cf73be43", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48562025-04-30T06:13:16.883ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
48572025-04-30T06:13:16.885ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e46e46a2-9304-49c4-b2d1-bcc29c0d177d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48582025-04-30T06:13:16.885ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
48592025-04-30T06:13:16.887ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "87cee6d7-8783-4c8c-b7d0-7e428d1ee1bb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48602025-04-30T06:13:16.887ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
48612025-04-30T06:13:16.888ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc67548f-8714-4bc3-b849-8de286134dee", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48622025-04-30T06:13:16.889ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
48632025-04-30T06:13:16.890ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee95cffe-c3fb-414c-b21b-f312d7526ca4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48642025-04-30T06:13:16.891ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
48652025-04-30T06:13:16.893ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36f22557-1828-445e-9cac-cf4439788175", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48662025-04-30T06:13:16.894ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
48672025-04-30T06:13:16.896ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "159f14ef-4aa3-448a-adea-ba20b29cfa01", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48682025-04-30T06:13:16.897ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
48692025-04-30T06:13:16.900ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bcc81fc-084f-43a2-b6e2-16ab6f68251b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48702025-04-30T06:13:16.900ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
48712025-04-30T06:13:16.902ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "24590d30-456a-4527-8887-c2005c1ee7d5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48722025-04-30T06:13:16.903ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
48732025-04-30T06:13:16.905ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
48742025-04-30T06:13:16.906ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2590f63e-c49c-4b5f-9d22-aebdc9954118", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48752025-04-30T06:13:16.906ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
48762025-04-30T06:13:16.909ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "34357fb8-87d3-4fa1-950f-4f7da7c837fa", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48772025-04-30T06:13:16.910ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
48782025-04-30T06:13:16.911ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e2d7f4d-8869-459f-a365-0d3ce1236a54", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48792025-04-30T06:13:16.911ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
48802025-04-30T06:13:16.912ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d1d58da-a890-4e6e-be41-c00ae4416a36", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48812025-04-30T06:13:16.912ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
48822025-04-30T06:13:16.914ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8c096d7-54f0-421c-8f83-20c8150b3b7f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48832025-04-30T06:13:16.914ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
48842025-04-30T06:13:16.916ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62344aa3-5632-4b46-aac9-60803cabd0f8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48852025-04-30T06:13:16.916ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
48862025-04-30T06:13:16.917ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bea8e328-1093-4d09-9c5d-11b47a0f3319", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48872025-04-30T06:13:16.917ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
48882025-04-30T06:13:16.919ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac55d3d8-8bc0-4e92-9fc6-717399a19ea7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48892025-04-30T06:13:16.919ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
48902025-04-30T06:13:16.921ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4df1c9d-aa25-4305-ab90-488d226bbcbe", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48912025-04-30T06:13:16.921ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
48922025-04-30T06:13:16.923ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6c72b0d-40e6-4e29-9b79-9915c9d8edcc", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48932025-04-30T06:13:16.923ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
48942025-04-30T06:13:16.925ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c863eef8-318e-422e-8218-12b17eb5acb7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48952025-04-30T06:13:16.925ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
48962025-04-30T06:13:16.927ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e1375ba-0602-4a35-a032-760d3d882dfb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48972025-04-30T06:13:16.927ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
48982025-04-30T06:13:16.929ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "01522656-1e41-497e-a5c3-bdfb8352a2ff", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
48992025-04-30T06:13:16.929ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
49002025-04-30T06:13:16.930ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8fbdd00-bd8e-4b2c-98e8-0759d10058f4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49012025-04-30T06:13:16.930ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
49022025-04-30T06:13:16.931ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e66719b1-1d0b-468b-95e7-460663aea4f5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49032025-04-30T06:13:16.932ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
49042025-04-30T06:13:16.933ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7b53080-5be4-4e47-936d-5259722e2bc2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49052025-04-30T06:13:16.933ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
49062025-04-30T06:13:16.935ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3de8021-ebb0-42dd-9ef2-a362efafd180", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49072025-04-30T06:13:16.935ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
49082025-04-30T06:13:16.936ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd220506-5162-4bca-8ff4-710e6d0b1193", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49092025-04-30T06:13:16.936ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
49102025-04-30T06:13:16.938ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6698bbea-3e4d-4996-9ae0-01f4a6e7c7a5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49112025-04-30T06:13:16.938ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
49122025-04-30T06:13:16.940ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d29902c-542f-483e-a6f8-ffa5fb96517a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49132025-04-30T06:13:16.940ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
49142025-04-30T06:13:16.942ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd80d95a-d649-4936-9c02-f1bc98be4c45", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49152025-04-30T06:13:16.942ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
49162025-04-30T06:13:16.944ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c68bca0c-c9e2-481f-8feb-3f6068ceda5e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49172025-04-30T06:13:16.944ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
49182025-04-30T06:13:16.946ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f59e7500-0cb4-473b-b7c4-aa2efc44b7e5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49192025-04-30T06:13:16.946ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
49202025-04-30T06:13:16.948ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "767a77ed-ce18-4742-9d03-2cf6e85220ff", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49212025-04-30T06:13:16.948ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
49222025-04-30T06:13:16.949ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ee6bf24-be62-48e9-b7d9-e36dd4b453e0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49232025-04-30T06:13:16.949ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
49242025-04-30T06:13:16.950ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac5c7193-1909-4a58-ae37-1bdf89c85b51", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49252025-04-30T06:13:16.950ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
49262025-04-30T06:13:16.951ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7295d8f0-789b-4161-ae38-8ef64959a761", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49272025-04-30T06:13:16.952ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
49282025-04-30T06:13:16.953ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "93f85c95-8dda-48ef-b9b9-b44db0fe56e7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49292025-04-30T06:13:16.953ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
49302025-04-30T06:13:16.954ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49312025-04-30T06:13:16.954ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
49322025-04-30T06:13:16.954ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
49332025-04-30T06:13:16.955ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
49342025-04-30T06:13:16.957ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
49352025-04-30T06:13:16.957ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49362025-04-30T06:13:16.957ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
49372025-04-30T06:13:16.957ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
49382025-04-30T06:13:16.959ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
49392025-04-30T06:13:16.959ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49402025-04-30T06:13:16.964ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
49412025-04-30T06:13:16.965ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
49422025-04-30T06:13:16.965ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49432025-04-30T06:13:16.965ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
49442025-04-30T06:13:16.965ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
49452025-04-30T06:13:16.966ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
49462025-04-30T06:13:16.968ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49472025-04-30T06:13:16.970ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49482025-04-30T06:13:16.975ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
49492025-04-30T06:13:16.975ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
49502025-04-30T06:13:16.975ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
49512025-04-30T06:13:16.975ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
49522025-04-30T06:13:16.975ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
49532025-04-30T06:13:16.976ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
49542025-04-30T06:13:16.976ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
49552025-04-30T06:13:16.977ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
49562025-04-30T06:13:16.979ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
49572025-04-30T06:13:16.979ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
49582025-04-30T06:13:16.980ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
49592025-04-30T06:13:17.021ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
49602025-04-30T06:13:17.043ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
49612025-04-30T06:13:17.043ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
49622025-04-30T06:13:17.044ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:55470 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
49632025-04-30T06:13:17.045ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:55470 req_id = 518ed90e-827b-499d-b9fc-bdf35d07108f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
49642025-04-30T06:13:17.046ZINFOomicron-dev (dropshot): request completed latency_us = 600 local_addr = [::1]:61881 method = GET remote_addr = [::1]:55470 req_id = 518ed90e-827b-499d-b9fc-bdf35d07108f response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
49652025-04-30T06:13:17.047ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "518ed90e-827b-499d-b9fc-bdf35d07108f", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
49662025-04-30T06:13:17.049ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
49672025-04-30T06:13:17.049ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
49682025-04-30T06:13:17.049ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
49692025-04-30T06:13:17.051ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
49702025-04-30T06:13:17.051ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49712025-04-30T06:13:17.051ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
49722025-04-30T06:13:17.051ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
49732025-04-30T06:13:17.052ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
49742025-04-30T06:13:17.054ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49752025-04-30T06:13:17.059ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
49762025-04-30T06:13:17.059ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
49772025-04-30T06:13:17.059ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
49782025-04-30T06:13:17.060ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
49792025-04-30T06:13:17.072ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
49802025-04-30T06:13:17.072ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49812025-04-30T06:13:17.072ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
49822025-04-30T06:13:17.072ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
49832025-04-30T06:13:17.073ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
49842025-04-30T06:13:17.075ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
49852025-04-30T06:13:17.081ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
49862025-04-30T06:13:17.081ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
49872025-04-30T06:13:17.082ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
49882025-04-30T06:13:17.082ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
49892025-04-30T06:13:17.083ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
49902025-04-30T06:13:17.084ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
49912025-04-30T06:13:17.084ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 299.18164ms iteration = 11 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
49922025-04-30T06:13:17.084ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 12 reason = Signaled
49932025-04-30T06:13:17.084ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
49942025-04-30T06:13:17.084ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
49952025-04-30T06:13:17.084ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
49962025-04-30T06:13:17.084ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
49972025-04-30T06:13:17.084ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
49982025-04-30T06:13:17.084ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
49992025-04-30T06:13:17.084ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
50002025-04-30T06:13:17.085ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
50012025-04-30T06:13:17.087ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50022025-04-30T06:13:17.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50032025-04-30T06:13:17.089ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
50042025-04-30T06:13:17.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50052025-04-30T06:13:17.090ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
50062025-04-30T06:13:17.090ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
50072025-04-30T06:13:17.091ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
50082025-04-30T06:13:17.091ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
50092025-04-30T06:13:17.091ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
50102025-04-30T06:13:17.094ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
50112025-04-30T06:13:17.094ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
50122025-04-30T06:13:17.094ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
50132025-04-30T06:13:17.094ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
50142025-04-30T06:13:17.095ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 33 9f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 3............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60889 req_id = a9408db4-8c5c-4f86-a2da-cc098753aba4
50152025-04-30T06:13:17.095ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13215,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60889 req_id = a9408db4-8c5c-4f86-a2da-cc098753aba4
50162025-04-30T06:13:17.095ZDEBGomicron-dev (store): query key key = _mgs._tcp
50172025-04-30T06:13:17.095ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
50182025-04-30T06:13:17.095ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60889 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = a9408db4-8c5c-4f86-a2da-cc098753aba4
50192025-04-30T06:13:17.096ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2461, tv_nsec: 895102362 } })
50202025-04-30T06:13:17.096ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
50212025-04-30T06:13:17.096ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ef e4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53335 req_id = 4dc52580-ffa0-47e3-a8cf-02fa12ee9419
50222025-04-30T06:13:17.096ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61412,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53335 req_id = 4dc52580-ffa0-47e3-a8cf-02fa12ee9419
50232025-04-30T06:13:17.097ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
50242025-04-30T06:13:17.097ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
50252025-04-30T06:13:17.097ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
50262025-04-30T06:13:17.097ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53335 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 4dc52580-ffa0-47e3-a8cf-02fa12ee9419
50272025-04-30T06:13:17.131ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
50282025-04-30T06:13:17.179ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
50292025-04-30T06:13:17.180ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 67 10 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c g............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54844 req_id = 5f3fdf06-f5f0-4370-b14a-4cd5750b8f34
50302025-04-30T06:13:17.180ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26384,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54844 req_id = 5f3fdf06-f5f0-4370-b14a-4cd5750b8f34
50312025-04-30T06:13:17.180ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
50322025-04-30T06:13:17.180ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 26384, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54844 req_id = 5f3fdf06-f5f0-4370-b14a-4cd5750b8f34
50332025-04-30T06:13:17.181ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
50342025-04-30T06:13:17.181ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
50352025-04-30T06:13:17.181ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
50362025-04-30T06:13:17.182ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
50372025-04-30T06:13:17.182ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
50382025-04-30T06:13:17.182ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
50392025-04-30T06:13:17.182ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
50402025-04-30T06:13:17.183ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
50412025-04-30T06:13:17.185ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "365e1862-090e-4cf9-bf91-2d4f384e78cf", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50422025-04-30T06:13:17.185ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
50432025-04-30T06:13:17.187ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb82ef90-72e0-426e-b39f-43c26981bb37", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50442025-04-30T06:13:17.187ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
50452025-04-30T06:13:17.190ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "18f272a9-9a3c-4427-a689-382a11080aec", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50462025-04-30T06:13:17.190ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
50472025-04-30T06:13:17.191ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
50482025-04-30T06:13:17.196ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "371977db-0a8a-4f79-bc39-7a7f6b7fff56", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50492025-04-30T06:13:17.196ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
50502025-04-30T06:13:17.196ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6bcbf226-3575-4911-8628-d12a7cd69a85", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50512025-04-30T06:13:17.196ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
50522025-04-30T06:13:17.196ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c2e5876-a5c2-44da-8ec0-37055e0bbb34", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50532025-04-30T06:13:17.197ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
50542025-04-30T06:13:17.198ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cac3232-e7c9-4d40-814c-fcf0a42e0418", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50552025-04-30T06:13:17.198ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
50562025-04-30T06:13:17.200ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc1ec068-4ef7-4947-a490-ed601be0d99f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50572025-04-30T06:13:17.200ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
50582025-04-30T06:13:17.201ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a5520cf-6e83-4d27-9881-75c459f97bd5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50592025-04-30T06:13:17.201ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
50602025-04-30T06:13:17.201ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
50612025-04-30T06:13:17.203ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "516dfd5b-fe7d-4ad6-a21e-09554e39d05e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50622025-04-30T06:13:17.203ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
50632025-04-30T06:13:17.205ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3b88080-b303-42d6-ba5b-e85b2e5c0eb6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50642025-04-30T06:13:17.205ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
50652025-04-30T06:13:17.206ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11074ce7-fe99-492c-bf99-9006c2d72024", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50662025-04-30T06:13:17.206ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
50672025-04-30T06:13:17.207ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
50682025-04-30T06:13:17.207ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cce2b1aa-934e-4d97-8a44-9574d6b38dba", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50692025-04-30T06:13:17.207ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
50702025-04-30T06:13:17.207ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
50712025-04-30T06:13:17.207ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
50722025-04-30T06:13:17.208ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
50732025-04-30T06:13:17.210ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "477bab4a-87fa-4918-9f57-415fdfd74a88", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50742025-04-30T06:13:17.210ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
50752025-04-30T06:13:17.212ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bf15f2d-f2fc-4215-acdc-ee39bdca9eba", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50762025-04-30T06:13:17.213ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
50772025-04-30T06:13:17.214ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "14027158-7886-49aa-b84d-b8d82cc6552a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50782025-04-30T06:13:17.215ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
50792025-04-30T06:13:17.216ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4af1d13-ea06-4495-85d6-90c5f26c2d53", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50802025-04-30T06:13:17.216ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
50812025-04-30T06:13:17.218ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5274f0a4-ec00-4ce8-b8cd-51eb32bc4355", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50822025-04-30T06:13:17.218ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
50832025-04-30T06:13:17.220ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "751747d2-47ed-401d-9cd3-659ea08408c3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50842025-04-30T06:13:17.220ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
50852025-04-30T06:13:17.220ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
50862025-04-30T06:13:17.221ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef55dbcd-9459-4824-a966-142c9075586f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50872025-04-30T06:13:17.221ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
50882025-04-30T06:13:17.221ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
50892025-04-30T06:13:17.222ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
50902025-04-30T06:13:17.223ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "803a7617-5b45-4829-802f-44bbec2fd64a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50912025-04-30T06:13:17.223ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
50922025-04-30T06:13:17.225ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2d0f0a4-fa7a-4b39-8151-0ec3d79389f4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50932025-04-30T06:13:17.226ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
50942025-04-30T06:13:17.227ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9b33d58-1e8c-4300-bee8-0fdec2808e94", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50952025-04-30T06:13:17.229ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
50962025-04-30T06:13:17.230ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fc66028-d15a-4da3-a85a-0690f3ad24d6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50972025-04-30T06:13:17.230ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
50982025-04-30T06:13:17.233ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e586216-5c8e-44ef-92e9-7d3cdec16dc5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
50992025-04-30T06:13:17.233ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
51002025-04-30T06:13:17.238ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d31f8f1b-4e11-4821-af9f-210d04527a3a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51012025-04-30T06:13:17.238ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
51022025-04-30T06:13:17.242ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d58a5e0-3753-4a34-a49b-ec1d495240e7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51032025-04-30T06:13:17.242ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
51042025-04-30T06:13:17.244ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "45ef751a-fb56-48af-8d3b-98f4500290f1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51052025-04-30T06:13:17.244ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
51062025-04-30T06:13:17.247ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "89eb6667-85fe-4b28-9df3-28c038ebf1e4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51072025-04-30T06:13:17.247ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
51082025-04-30T06:13:17.249ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "848cf9d1-565c-4c21-bc5e-7961eefd0f08", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51092025-04-30T06:13:17.249ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
51102025-04-30T06:13:17.250ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9c4b729-c91f-440c-9e03-d0a8da43d559", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51112025-04-30T06:13:17.250ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
51122025-04-30T06:13:17.252ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd26746b-8ed5-4297-aa64-451832c917a2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51132025-04-30T06:13:17.252ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
51142025-04-30T06:13:17.254ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5223ce95-5796-4e82-91bf-f2946235f8e6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51152025-04-30T06:13:17.254ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
51162025-04-30T06:13:17.255ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a184ff3e-8fbc-4bcf-b484-53e3315b11a0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51172025-04-30T06:13:17.255ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
51182025-04-30T06:13:17.256ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9585e904-4895-46e6-9236-013aadc6e9a5", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51192025-04-30T06:13:17.256ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
51202025-04-30T06:13:17.258ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7b12b4be-1c38-450b-8be1-c8690100a40c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51212025-04-30T06:13:17.258ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
51222025-04-30T06:13:17.260ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
51232025-04-30T06:13:17.261ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6f3e9ae-2804-424d-9833-763c31ac515d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51242025-04-30T06:13:17.261ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
51252025-04-30T06:13:17.263ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a18abd1d-ef91-4ce7-8243-79a2cab68bc3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51262025-04-30T06:13:17.264ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
51272025-04-30T06:13:17.265ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb81ce7d-da47-4507-930b-5b8a1298aa65", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51282025-04-30T06:13:17.266ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
51292025-04-30T06:13:17.267ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "077df56c-982a-4925-af5c-2d60b50b98d3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51302025-04-30T06:13:17.268ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
51312025-04-30T06:13:17.269ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cad6bcd1-71d5-423b-97da-ae405d531561", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51322025-04-30T06:13:17.269ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
51332025-04-30T06:13:17.270ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "51823a10-dec3-4be2-ae92-468bffb9aae6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51342025-04-30T06:13:17.271ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
51352025-04-30T06:13:17.272ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "06541a88-4860-4ff4-a16d-79970202b8b0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51362025-04-30T06:13:17.272ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
51372025-04-30T06:13:17.273ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d2e58c2-0bdb-41d1-a19a-229a953c406c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51382025-04-30T06:13:17.273ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
51392025-04-30T06:13:17.275ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6d81cfc-002a-4663-a2c8-fc492aefb6ce", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51402025-04-30T06:13:17.275ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
51412025-04-30T06:13:17.275ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51422025-04-30T06:13:17.275ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
51432025-04-30T06:13:17.275ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51442025-04-30T06:13:17.276ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
51452025-04-30T06:13:17.279ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51462025-04-30T06:13:17.282ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
51472025-04-30T06:13:17.282ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
51482025-04-30T06:13:17.282ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51492025-04-30T06:13:17.282ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
51502025-04-30T06:13:17.282ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51512025-04-30T06:13:17.283ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
51522025-04-30T06:13:17.286ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51532025-04-30T06:13:17.290ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
51542025-04-30T06:13:17.290ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
51552025-04-30T06:13:17.290ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51562025-04-30T06:13:17.290ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
51572025-04-30T06:13:17.311ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
51582025-04-30T06:13:17.312ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51592025-04-30T06:13:17.312ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
51602025-04-30T06:13:17.312ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
51612025-04-30T06:13:17.312ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
51622025-04-30T06:13:17.329ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
51632025-04-30T06:13:17.329ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
51642025-04-30T06:13:17.330ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:41950 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
51652025-04-30T06:13:17.330ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:41950 req_id = b0bde07c-5057-4eb9-b3ca-fd8665701fca sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
51662025-04-30T06:13:17.331ZINFOomicron-dev (dropshot): request completed latency_us = 783 local_addr = [::1]:61881 method = GET remote_addr = [::1]:41950 req_id = b0bde07c-5057-4eb9-b3ca-fd8665701fca response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
51672025-04-30T06:13:17.331ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0bde07c-5057-4eb9-b3ca-fd8665701fca", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
51682025-04-30T06:13:17.332ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51692025-04-30T06:13:17.334ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
51702025-04-30T06:13:17.334ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
51712025-04-30T06:13:17.334ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
51722025-04-30T06:13:17.336ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
51732025-04-30T06:13:17.336ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51742025-04-30T06:13:17.336ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
51752025-04-30T06:13:17.336ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51762025-04-30T06:13:17.337ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
51772025-04-30T06:13:17.340ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
51782025-04-30T06:13:17.340ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
51792025-04-30T06:13:17.340ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
51802025-04-30T06:13:17.340ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
51812025-04-30T06:13:17.342ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
51822025-04-30T06:13:17.344ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
51832025-04-30T06:13:17.344ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
51842025-04-30T06:13:17.345ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
51852025-04-30T06:13:17.348ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
51862025-04-30T06:13:17.348ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
51872025-04-30T06:13:17.348ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51882025-04-30T06:13:17.349ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
51892025-04-30T06:13:17.390ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
51902025-04-30T06:13:17.390ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
51912025-04-30T06:13:17.390ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
51922025-04-30T06:13:17.390ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 306.498098ms iteration = 12 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
51932025-04-30T06:13:17.390ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 13 reason = Signaled
51942025-04-30T06:13:17.390ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
51952025-04-30T06:13:17.390ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
51962025-04-30T06:13:17.390ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
51972025-04-30T06:13:17.390ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
51982025-04-30T06:13:17.391ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
51992025-04-30T06:13:17.394ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52002025-04-30T06:13:17.398ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
52012025-04-30T06:13:17.398ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
52022025-04-30T06:13:17.398ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
52032025-04-30T06:13:17.398ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
52042025-04-30T06:13:17.402ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
52052025-04-30T06:13:17.402ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
52062025-04-30T06:13:17.403ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d0 db 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53402 req_id = ef3f09ac-df9d-47bd-b879-55274c1616a9
52072025-04-30T06:13:17.403ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53467,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53402 req_id = ef3f09ac-df9d-47bd-b879-55274c1616a9
52082025-04-30T06:13:17.403ZDEBGomicron-dev (store): query key key = _mgs._tcp
52092025-04-30T06:13:17.403ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
52102025-04-30T06:13:17.403ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53402 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = ef3f09ac-df9d-47bd-b879-55274c1616a9
52112025-04-30T06:13:17.404ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2462, tv_nsec: 203133774 } })
52122025-04-30T06:13:17.404ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
52132025-04-30T06:13:17.405ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 60 9d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e `...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59840 req_id = 747532af-7910-4bc6-86f6-c047da71d21f
52142025-04-30T06:13:17.405ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24733,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59840 req_id = 747532af-7910-4bc6-86f6-c047da71d21f
52152025-04-30T06:13:17.405ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
52162025-04-30T06:13:17.405ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59840 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 747532af-7910-4bc6-86f6-c047da71d21f
52172025-04-30T06:13:17.441ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
52182025-04-30T06:13:17.442ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
52192025-04-30T06:13:17.442ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
52202025-04-30T06:13:17.442ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
52212025-04-30T06:13:17.442ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
52222025-04-30T06:13:17.446ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
52232025-04-30T06:13:17.446ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
52242025-04-30T06:13:17.446ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 15 00 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:56834 req_id = 71227c56-8d2b-4b8e-b0ca-0b005c3f6c06
52252025-04-30T06:13:17.446ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5376,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56834 req_id = 71227c56-8d2b-4b8e-b0ca-0b005c3f6c06
52262025-04-30T06:13:17.446ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
52272025-04-30T06:13:17.447ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5376, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56834 req_id = 71227c56-8d2b-4b8e-b0ca-0b005c3f6c06
52282025-04-30T06:13:17.447ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
52292025-04-30T06:13:17.447ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
52302025-04-30T06:13:17.447ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
52312025-04-30T06:13:17.449ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "d136ff71-227c-4adc-bccf-f1622760daf6", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52322025-04-30T06:13:17.449ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
52332025-04-30T06:13:17.450ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
52342025-04-30T06:13:17.450ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd6b8bd8-3ac6-42b8-b8db-ffdb446fb8c8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52352025-04-30T06:13:17.450ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
52362025-04-30T06:13:17.451ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
52372025-04-30T06:13:17.451ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
52382025-04-30T06:13:17.451ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
52392025-04-30T06:13:17.453ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "39e5c0bc-8184-49b0-abe4-138b3d25bdd6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52402025-04-30T06:13:17.454ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
52412025-04-30T06:13:17.454ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
52422025-04-30T06:13:17.454ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
52432025-04-30T06:13:17.454ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
52442025-04-30T06:13:17.456ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "20978420-829c-463c-97e6-a7693662c08a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52452025-04-30T06:13:17.456ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
52462025-04-30T06:13:17.460ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "38ed9c01-909c-40bb-a424-4cff1db6f6fd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52472025-04-30T06:13:17.461ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
52482025-04-30T06:13:17.461ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a6c07d0-9fa7-4c7c-9125-fb1b4f475123", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52492025-04-30T06:13:17.461ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
52502025-04-30T06:13:17.463ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe2f32e5-2427-4a31-a628-6701042900f6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52512025-04-30T06:13:17.463ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
52522025-04-30T06:13:17.466ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "920f5c71-5cd8-47e6-92fd-564a2fbbdefd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52532025-04-30T06:13:17.467ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
52542025-04-30T06:13:17.467ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "371b3561-8754-44c3-941f-c75bb5e03bc3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52552025-04-30T06:13:17.468ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
52562025-04-30T06:13:17.471ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6f1dc4b-8f30-4bf1-ada8-83038cb849da", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52572025-04-30T06:13:17.471ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
52582025-04-30T06:13:17.473ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a60dd23-b375-4c3b-a7f0-5bb70a588994", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52592025-04-30T06:13:17.474ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
52602025-04-30T06:13:17.475ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "89f51e2c-2bae-41d9-914e-3342d6dcd866", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52612025-04-30T06:13:17.475ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
52622025-04-30T06:13:17.478ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dccd6dd-3901-41fe-8bdd-bca16ac430d2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52632025-04-30T06:13:17.478ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
52642025-04-30T06:13:17.484ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa6d6c43-939d-4b42-be03-2157f514507a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52652025-04-30T06:13:17.484ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
52662025-04-30T06:13:17.487ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a018a552-b5bd-4730-8e07-d7ea34766364", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52672025-04-30T06:13:17.487ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
52682025-04-30T06:13:17.489ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
52692025-04-30T06:13:17.491ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "903365f8-8265-4e91-aa4a-f02a3ab09e45", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52702025-04-30T06:13:17.491ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
52712025-04-30T06:13:17.495ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb7aaa6d-64ab-4ec9-8467-3be1177d6660", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52722025-04-30T06:13:17.495ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
52732025-04-30T06:13:17.498ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7dd612dc-51ef-4976-8b82-3cac653434e0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52742025-04-30T06:13:17.498ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
52752025-04-30T06:13:17.500ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "18bc83a6-5a53-4bc7-b7dc-80c821e27837", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52762025-04-30T06:13:17.500ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
52772025-04-30T06:13:17.502ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b09d86c-4bc3-4f5c-9120-562c34e55276", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52782025-04-30T06:13:17.502ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
52792025-04-30T06:13:17.503ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f616992-3dd1-4b1b-9b0d-01822e3653af", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52802025-04-30T06:13:17.504ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
52812025-04-30T06:13:17.505ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e5eabc3-3d7d-4ddb-97f7-4397a7e069c2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
52822025-04-30T06:13:17.505ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
52832025-04-30T06:13:17.507ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "067d4fb4-e596-413e-be5d-708b07b2c898", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52842025-04-30T06:13:17.507ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
52852025-04-30T06:13:17.508ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d57f0ebc-9b71-4fda-b5a9-0cd23039b695", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52862025-04-30T06:13:17.508ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
52872025-04-30T06:13:17.510ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "75214b61-dcfb-425d-894e-cc3a0f19d26d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52882025-04-30T06:13:17.510ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
52892025-04-30T06:13:17.512ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "44799ef1-db71-4d73-9dc5-765cdba14516", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52902025-04-30T06:13:17.512ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
52912025-04-30T06:13:17.514ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4536926-e9df-4216-9ab5-f4c5d68a826f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52922025-04-30T06:13:17.514ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
52932025-04-30T06:13:17.516ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3303fe7a-7223-4e7c-ad70-d40f308e31ed", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52942025-04-30T06:13:17.516ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
52952025-04-30T06:13:17.519ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f963ae6-52ab-4531-acd6-70ff0ae49689", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52962025-04-30T06:13:17.519ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
52972025-04-30T06:13:17.519ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1eed492c-bd05-45c8-9abe-97c3f7f24f67", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
52982025-04-30T06:13:17.519ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
52992025-04-30T06:13:17.520ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45e38bd6-9b4f-4208-9961-e8761fc081e9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
53002025-04-30T06:13:17.520ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
53012025-04-30T06:13:17.521ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5539d419-60cf-4e0e-83ee-7e42c59a15f3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53022025-04-30T06:13:17.522ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
53032025-04-30T06:13:17.523ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6c71ef5-b92f-42e1-a225-694f015e937f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53042025-04-30T06:13:17.523ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
53052025-04-30T06:13:17.525ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9bcb1ad-2f6f-449e-ae5e-bee168c7fc2e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53062025-04-30T06:13:17.525ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
53072025-04-30T06:13:17.527ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9a476ae-dd9d-44cc-a783-e56238a3a581", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53082025-04-30T06:13:17.527ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
53092025-04-30T06:13:17.530ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf6e62e4-6ac9-4eda-a106-fa35ed510830", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53102025-04-30T06:13:17.530ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
53112025-04-30T06:13:17.533ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d15f8da9-1827-46eb-b7ac-a92a83fd0de4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53122025-04-30T06:13:17.533ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
53132025-04-30T06:13:17.535ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7161adc8-d87d-431c-8a1b-90d7ac023256", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53142025-04-30T06:13:17.535ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
53152025-04-30T06:13:17.537ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f71d2ad2-b382-49ba-b3b5-28a3c0870328", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53162025-04-30T06:13:17.538ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
53172025-04-30T06:13:17.539ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd77969b-6aaf-4a4f-a0eb-b991fae20da7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53182025-04-30T06:13:17.540ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
53192025-04-30T06:13:17.540ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
53202025-04-30T06:13:17.540ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53212025-04-30T06:13:17.540ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
53222025-04-30T06:13:17.540ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
53232025-04-30T06:13:17.541ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
53242025-04-30T06:13:17.543ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d8994ae-07ca-4b87-90d2-9e214034e7e8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
53252025-04-30T06:13:17.543ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
53262025-04-30T06:13:17.544ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53272025-04-30T06:13:17.545ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "365e535b-8238-4a0e-9c8b-6017ba09aeb3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53282025-04-30T06:13:17.545ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
53292025-04-30T06:13:17.546ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c32beee-5b1c-48ab-980a-b1e19a67384b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53302025-04-30T06:13:17.546ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
53312025-04-30T06:13:17.548ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a007d3af-803c-4e9e-b7db-a3ffa376b181", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53322025-04-30T06:13:17.548ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
53332025-04-30T06:13:17.549ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "266dc54f-09b0-41af-9964-634e30a59619", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:16 GMT"} })
53342025-04-30T06:13:17.549ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
53352025-04-30T06:13:17.549ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53362025-04-30T06:13:17.549ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
53372025-04-30T06:13:17.549ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53382025-04-30T06:13:17.549ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
53392025-04-30T06:13:17.549ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
53402025-04-30T06:13:17.549ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
53412025-04-30T06:13:17.549ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
53422025-04-30T06:13:17.550ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
53432025-04-30T06:13:17.552ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
53442025-04-30T06:13:17.552ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
53452025-04-30T06:13:17.554ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53462025-04-30T06:13:17.554ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
53472025-04-30T06:13:17.560ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
53482025-04-30T06:13:17.560ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
53492025-04-30T06:13:17.560ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53502025-04-30T06:13:17.560ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
53512025-04-30T06:13:17.560ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53522025-04-30T06:13:17.561ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
53532025-04-30T06:13:17.564ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53542025-04-30T06:13:17.569ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
53552025-04-30T06:13:17.569ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
53562025-04-30T06:13:17.569ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53572025-04-30T06:13:17.570ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
53582025-04-30T06:13:17.611ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
53592025-04-30T06:13:17.623ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
53602025-04-30T06:13:17.623ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
53612025-04-30T06:13:17.624ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:54958 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
53622025-04-30T06:13:17.624ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:54958 req_id = 0b292bfd-c6b3-4f20-a9f1-614da2b07e31 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
53632025-04-30T06:13:17.625ZINFOomicron-dev (dropshot): request completed latency_us = 585 local_addr = [::1]:61881 method = GET remote_addr = [::1]:54958 req_id = 0b292bfd-c6b3-4f20-a9f1-614da2b07e31 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
53642025-04-30T06:13:17.625ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b292bfd-c6b3-4f20-a9f1-614da2b07e31", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
53652025-04-30T06:13:17.627ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
53662025-04-30T06:13:17.628ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
53672025-04-30T06:13:17.628ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
53682025-04-30T06:13:17.631ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
53692025-04-30T06:13:17.631ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53702025-04-30T06:13:17.631ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
53712025-04-30T06:13:17.631ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53722025-04-30T06:13:17.632ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
53732025-04-30T06:13:17.636ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53742025-04-30T06:13:17.641ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
53752025-04-30T06:13:17.641ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
53762025-04-30T06:13:17.641ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53772025-04-30T06:13:17.641ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
53782025-04-30T06:13:17.662ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
53792025-04-30T06:13:17.662ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53802025-04-30T06:13:17.663ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
53812025-04-30T06:13:17.663ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
53822025-04-30T06:13:17.664ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
53832025-04-30T06:13:17.667ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
53842025-04-30T06:13:17.673ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
53852025-04-30T06:13:17.673ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
53862025-04-30T06:13:17.673ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
53872025-04-30T06:13:17.674ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
53882025-04-30T06:13:17.675ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
53892025-04-30T06:13:17.676ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
53902025-04-30T06:13:17.676ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 285.693866ms iteration = 13 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
53912025-04-30T06:13:17.676ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 14 reason = Signaled
53922025-04-30T06:13:17.676ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
53932025-04-30T06:13:17.677ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
53942025-04-30T06:13:17.677ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
53952025-04-30T06:13:17.677ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
53962025-04-30T06:13:17.678ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
53972025-04-30T06:13:17.682ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
53982025-04-30T06:13:17.682ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
53992025-04-30T06:13:17.682ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54002025-04-30T06:13:17.683ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
54012025-04-30T06:13:17.687ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
54022025-04-30T06:13:17.687ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
54032025-04-30T06:13:17.687ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
54042025-04-30T06:13:17.687ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
54052025-04-30T06:13:17.692ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
54062025-04-30T06:13:17.692ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
54072025-04-30T06:13:17.694ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 5a a1 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 Z............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56860 req_id = 821bce2e-b6e2-487c-861d-f975758bdda3
54082025-04-30T06:13:17.694ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23201,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56860 req_id = 821bce2e-b6e2-487c-861d-f975758bdda3
54092025-04-30T06:13:17.695ZDEBGomicron-dev (store): query key key = _mgs._tcp
54102025-04-30T06:13:17.695ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
54112025-04-30T06:13:17.695ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56860 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 821bce2e-b6e2-487c-861d-f975758bdda3
54122025-04-30T06:13:17.695ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2462, tv_nsec: 494812916 } })
54132025-04-30T06:13:17.696ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
54142025-04-30T06:13:17.697ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a3 60 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .`..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60447 req_id = 14c64735-576a-4b0e-8feb-20c95979591c
54152025-04-30T06:13:17.697ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41824,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60447 req_id = 14c64735-576a-4b0e-8feb-20c95979591c
54162025-04-30T06:13:17.698ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
54172025-04-30T06:13:17.698ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60447 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 14c64735-576a-4b0e-8feb-20c95979591c
54182025-04-30T06:13:17.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54192025-04-30T06:13:17.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54202025-04-30T06:13:17.721ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54212025-04-30T06:13:17.722ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54222025-04-30T06:13:17.722ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54232025-04-30T06:13:17.722ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
54242025-04-30T06:13:17.734ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
54252025-04-30T06:13:17.755ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
54262025-04-30T06:13:17.755ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 48 97 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c H............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50071 req_id = 60004342-5065-42e9-9dd9-315ded4afe9c
54272025-04-30T06:13:17.755ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 18583,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50071 req_id = 60004342-5065-42e9-9dd9-315ded4afe9c
54282025-04-30T06:13:17.756ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
54292025-04-30T06:13:17.756ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 18583, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50071 req_id = 60004342-5065-42e9-9dd9-315ded4afe9c
54302025-04-30T06:13:17.756ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
54312025-04-30T06:13:17.756ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
54322025-04-30T06:13:17.756ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
54332025-04-30T06:13:17.759ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "3eca43c9-1dbf-4ec6-8bf2-c1b3ce7d92ca", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54342025-04-30T06:13:17.759ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
54352025-04-30T06:13:17.762ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f36c6a8-346d-46d3-a614-7c0bd6fb39ec", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54362025-04-30T06:13:17.762ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
54372025-04-30T06:13:17.765ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a81a983-9fdf-4636-8b78-a2e8ea7da060", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54382025-04-30T06:13:17.765ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
54392025-04-30T06:13:17.767ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "532d0c43-ec32-4a36-b009-5b90bf73926d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54402025-04-30T06:13:17.767ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
54412025-04-30T06:13:17.769ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3aa1993c-1a0f-454e-9453-b0791a57404a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54422025-04-30T06:13:17.769ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
54432025-04-30T06:13:17.771ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8d47861-642e-4b34-be8a-aa05bd85140e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54442025-04-30T06:13:17.771ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
54452025-04-30T06:13:17.774ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "26a827e0-8d60-4552-83af-0ea8cea4b9ce", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54462025-04-30T06:13:17.774ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
54472025-04-30T06:13:17.775ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3970e19-32e3-4831-a066-a19db868ae62", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54482025-04-30T06:13:17.776ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
54492025-04-30T06:13:17.776ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7bce1df-4088-4090-9955-92a5f8d262c7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54502025-04-30T06:13:17.777ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
54512025-04-30T06:13:17.779ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29001e5f-decc-4193-8700-afa964338652", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54522025-04-30T06:13:17.779ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
54532025-04-30T06:13:17.780ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11809eb2-322f-49ee-870d-703e487a327e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54542025-04-30T06:13:17.780ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
54552025-04-30T06:13:17.781ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4de671a-a224-4b7d-948a-fa364e30899e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54562025-04-30T06:13:17.781ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
54572025-04-30T06:13:17.783ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "20cebe85-4122-4434-9c4e-0fefb49cbd03", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54582025-04-30T06:13:17.783ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
54592025-04-30T06:13:17.784ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a8e5093-9f36-41be-8300-80a0836d0af2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54602025-04-30T06:13:17.784ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
54612025-04-30T06:13:17.785ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
54622025-04-30T06:13:17.785ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
54632025-04-30T06:13:17.785ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
54642025-04-30T06:13:17.785ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
54652025-04-30T06:13:17.786ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
54662025-04-30T06:13:17.786ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b05831cd-814f-422a-96d4-da692cfe6a70", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54672025-04-30T06:13:17.786ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
54682025-04-30T06:13:17.788ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
54692025-04-30T06:13:17.789ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ada3c9d2-3470-453d-af57-a9978f1e0bd4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54702025-04-30T06:13:17.789ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
54712025-04-30T06:13:17.793ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "19e4840a-448f-47c3-92c8-580aa1915da1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54722025-04-30T06:13:17.793ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
54732025-04-30T06:13:17.794ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
54742025-04-30T06:13:17.794ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
54752025-04-30T06:13:17.794ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
54762025-04-30T06:13:17.795ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
54772025-04-30T06:13:17.795ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d145550c-0600-40f8-8a31-0e45eabb0f56", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54782025-04-30T06:13:17.795ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
54792025-04-30T06:13:17.797ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
54802025-04-30T06:13:17.797ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
54812025-04-30T06:13:17.798ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
54822025-04-30T06:13:17.800ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8d43d2c-aeb0-4bf3-bab6-9c1b3ff719b5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54832025-04-30T06:13:17.800ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
54842025-04-30T06:13:17.801ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7ce1b0b-31ce-4d6a-b483-6e7223f6686e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54852025-04-30T06:13:17.802ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
54862025-04-30T06:13:17.805ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43e046ae-b7be-4527-b6e1-b2193ef0a1c8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54872025-04-30T06:13:17.805ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
54882025-04-30T06:13:17.806ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ae6f8d-5ec4-4e18-b224-8d350aa23bb8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54892025-04-30T06:13:17.806ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
54902025-04-30T06:13:17.809ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba6afd4f-cb6d-4f2b-b8cc-492905dcddd0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54912025-04-30T06:13:17.809ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
54922025-04-30T06:13:17.812ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e3d06e0-8111-4e09-af2f-45a33d24fea1", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54932025-04-30T06:13:17.812ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
54942025-04-30T06:13:17.815ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1adae568-3d81-4744-a453-3a92c3f8e387", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54952025-04-30T06:13:17.815ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
54962025-04-30T06:13:17.821ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "353cee48-f1c8-405e-9e3b-f69b3ed008b8", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54972025-04-30T06:13:17.823ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
54982025-04-30T06:13:17.827ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "37346fd1-e1d2-43bb-a9d6-c641d141e31a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
54992025-04-30T06:13:17.828ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
55002025-04-30T06:13:17.832ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92f07a5f-e4ec-4362-a2c1-66b621a56f8a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55012025-04-30T06:13:17.832ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
55022025-04-30T06:13:17.835ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a9a2ca9-ff61-4ae4-af7d-f7484375372f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55032025-04-30T06:13:17.835ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
55042025-04-30T06:13:17.838ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9bc1c8a-3ed1-4483-9aaf-f276ddc082de", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55052025-04-30T06:13:17.838ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
55062025-04-30T06:13:17.839ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99d7d556-72c1-429a-88b8-19c93580e3d5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55072025-04-30T06:13:17.839ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
55082025-04-30T06:13:17.840ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f46bf1d-d537-4010-b4f6-ed393895d993", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55092025-04-30T06:13:17.840ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
55102025-04-30T06:13:17.841ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
55112025-04-30T06:13:17.841ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "584eed15-214d-42dc-b85e-be168cdf1852", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55122025-04-30T06:13:17.841ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
55132025-04-30T06:13:17.842ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc6b3cfa-6752-444a-9c21-7839267f3af3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55142025-04-30T06:13:17.843ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
55152025-04-30T06:13:17.844ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3c6131e-1721-47ca-bc8c-cb5d869e9ef9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55162025-04-30T06:13:17.844ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
55172025-04-30T06:13:17.846ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c23aa615-cdd1-4e2a-b0af-689b34dc1b1b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55182025-04-30T06:13:17.846ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
55192025-04-30T06:13:17.849ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ead3897-20c4-4f2b-a339-aba96f0cb34b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55202025-04-30T06:13:17.849ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
55212025-04-30T06:13:17.851ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eefae904-8cc7-4f99-a905-2f0af1083af0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55222025-04-30T06:13:17.851ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
55232025-04-30T06:13:17.854ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b7e963c-eaf4-467e-ab46-99fa18576952", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55242025-04-30T06:13:17.855ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
55252025-04-30T06:13:17.856ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2246c5c0-b0a2-40d5-bc5e-fd68b3ca24a2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55262025-04-30T06:13:17.857ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
55272025-04-30T06:13:17.859ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccd10507-2b5c-47f1-badf-1be3bca4e5ee", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55282025-04-30T06:13:17.859ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
55292025-04-30T06:13:17.860ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea9502f9-1ac5-498a-9b20-7f62e58d2e8f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55302025-04-30T06:13:17.860ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
55312025-04-30T06:13:17.861ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b19d18cd-1a62-4fb1-b774-4e02c8842b3d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55322025-04-30T06:13:17.861ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
55332025-04-30T06:13:17.862ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "50df2f48-752a-4314-8b55-0751151c4dfd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55342025-04-30T06:13:17.862ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
55352025-04-30T06:13:17.864ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "74abc83d-da5a-412f-863e-bb665de1b129", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55362025-04-30T06:13:17.865ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
55372025-04-30T06:13:17.865ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55382025-04-30T06:13:17.865ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55392025-04-30T06:13:17.865ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55402025-04-30T06:13:17.866ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55412025-04-30T06:13:17.869ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55422025-04-30T06:13:17.876ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
55432025-04-30T06:13:17.876ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
55442025-04-30T06:13:17.876ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55452025-04-30T06:13:17.876ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55462025-04-30T06:13:17.876ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55472025-04-30T06:13:17.876ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55482025-04-30T06:13:17.880ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55492025-04-30T06:13:17.883ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
55502025-04-30T06:13:17.883ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55512025-04-30T06:13:17.883ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55522025-04-30T06:13:17.884ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55532025-04-30T06:13:17.892ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
55542025-04-30T06:13:17.892ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55552025-04-30T06:13:17.893ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
55562025-04-30T06:13:17.893ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
55572025-04-30T06:13:17.893ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55582025-04-30T06:13:17.942ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
55592025-04-30T06:13:17.942ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
55602025-04-30T06:13:17.943ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:65243 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
55612025-04-30T06:13:17.943ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:65243 req_id = ed93156e-6084-4531-b13d-be840b83a929 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
55622025-04-30T06:13:17.944ZINFOomicron-dev (dropshot): request completed latency_us = 547 local_addr = [::1]:61881 method = GET remote_addr = [::1]:65243 req_id = ed93156e-6084-4531-b13d-be840b83a929 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
55632025-04-30T06:13:17.944ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed93156e-6084-4531-b13d-be840b83a929", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
55642025-04-30T06:13:17.946ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55652025-04-30T06:13:17.947ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
55662025-04-30T06:13:17.947ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
55672025-04-30T06:13:17.947ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
55682025-04-30T06:13:17.949ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
55692025-04-30T06:13:17.949ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55702025-04-30T06:13:17.949ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55712025-04-30T06:13:17.949ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55722025-04-30T06:13:17.950ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55732025-04-30T06:13:17.953ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55742025-04-30T06:13:17.953ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
55752025-04-30T06:13:17.953ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
55762025-04-30T06:13:17.953ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
55772025-04-30T06:13:17.954ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55782025-04-30T06:13:17.957ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
55792025-04-30T06:13:17.957ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55802025-04-30T06:13:17.957ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55812025-04-30T06:13:17.958ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55822025-04-30T06:13:17.960ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
55832025-04-30T06:13:17.960ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
55842025-04-30T06:13:17.962ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
55852025-04-30T06:13:18.042ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
55862025-04-30T06:13:18.046ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
55872025-04-30T06:13:18.047ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
55882025-04-30T06:13:18.048ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 370.850615ms iteration = 14 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
55892025-04-30T06:13:18.048ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 15 reason = Signaled
55902025-04-30T06:13:18.048ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
55912025-04-30T06:13:18.048ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
55922025-04-30T06:13:18.048ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55932025-04-30T06:13:18.048ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55942025-04-30T06:13:18.050ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
55952025-04-30T06:13:18.055ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
55962025-04-30T06:13:18.062ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
55972025-04-30T06:13:18.062ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
55982025-04-30T06:13:18.062ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
55992025-04-30T06:13:18.062ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
56002025-04-30T06:13:18.068ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
56012025-04-30T06:13:18.068ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
56022025-04-30T06:13:18.069ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 50 ad 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 P............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:65032 req_id = 64edf452-e294-4891-9713-59f637e04fca
56032025-04-30T06:13:18.069ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20653,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65032 req_id = 64edf452-e294-4891-9713-59f637e04fca
56042025-04-30T06:13:18.069ZDEBGomicron-dev (store): query key key = _mgs._tcp
56052025-04-30T06:13:18.069ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
56062025-04-30T06:13:18.069ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65032 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 64edf452-e294-4891-9713-59f637e04fca
56072025-04-30T06:13:18.070ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2462, tv_nsec: 869275568 } })
56082025-04-30T06:13:18.070ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
56092025-04-30T06:13:18.070ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 5c 04 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e \\...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58103 req_id = 59ed10c9-48e8-44fd-afa8-f570717dfc61
56102025-04-30T06:13:18.070ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23556,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58103 req_id = 59ed10c9-48e8-44fd-afa8-f570717dfc61
56112025-04-30T06:13:18.071ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
56122025-04-30T06:13:18.071ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58103 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 59ed10c9-48e8-44fd-afa8-f570717dfc61
56132025-04-30T06:13:18.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
56142025-04-30T06:13:18.089ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
56152025-04-30T06:13:18.090ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
56162025-04-30T06:13:18.092ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
56172025-04-30T06:13:18.095ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
56182025-04-30T06:13:18.095ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
56192025-04-30T06:13:18.095ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
56202025-04-30T06:13:18.095ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
56212025-04-30T06:13:18.096ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
56222025-04-30T06:13:18.096ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
56232025-04-30T06:13:18.097ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
56242025-04-30T06:13:18.100ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
56252025-04-30T06:13:18.100ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
56262025-04-30T06:13:18.104ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
56272025-04-30T06:13:18.111ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
56282025-04-30T06:13:18.111ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
56292025-04-30T06:13:18.111ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
56302025-04-30T06:13:18.111ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
56312025-04-30T06:13:18.114ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
56322025-04-30T06:13:18.114ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
56332025-04-30T06:13:18.114ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
56342025-04-30T06:13:18.115ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
56352025-04-30T06:13:18.116ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: b5 46 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .F..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51644 req_id = 9e2474b3-b00f-4828-ad91-91b124243c64
56362025-04-30T06:13:18.116ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46406,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51644 req_id = 9e2474b3-b00f-4828-ad91-91b124243c64
56372025-04-30T06:13:18.117ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
56382025-04-30T06:13:18.117ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 46406, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51644 req_id = 9e2474b3-b00f-4828-ad91-91b124243c64
56392025-04-30T06:13:18.117ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
56402025-04-30T06:13:18.117ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
56412025-04-30T06:13:18.117ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
56422025-04-30T06:13:18.122ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "146951ff-0340-4060-99b7-048425fff865", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56432025-04-30T06:13:18.122ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
56442025-04-30T06:13:18.124ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16ce1328-82e0-4efe-bc0e-ed3310b641bb", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56452025-04-30T06:13:18.124ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
56462025-04-30T06:13:18.126ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b93592f-5c6e-4989-800a-ae965b36b6fe", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56472025-04-30T06:13:18.127ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
56482025-04-30T06:13:18.129ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b1a5617-646f-42c6-9dec-b15c965a4abe", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56492025-04-30T06:13:18.129ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
56502025-04-30T06:13:18.131ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "de1e1a7c-df70-4b6e-8f64-420483f439e3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56512025-04-30T06:13:18.131ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
56522025-04-30T06:13:18.133ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5be1add-eed6-449d-bad6-873343555abc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56532025-04-30T06:13:18.133ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
56542025-04-30T06:13:18.136ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a93e101d-06e9-42f6-918c-2e773a0ba3a7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56552025-04-30T06:13:18.136ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
56562025-04-30T06:13:18.139ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3be6b492-132b-4003-abda-b8173aaef754", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56572025-04-30T06:13:18.139ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
56582025-04-30T06:13:18.140ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31be4ddc-a1f2-4734-8053-d72a9d121fed", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56592025-04-30T06:13:18.140ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
56602025-04-30T06:13:18.143ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "399b2e25-7d27-47f6-bd2d-bfcd89a5a768", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56612025-04-30T06:13:18.143ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
56622025-04-30T06:13:18.143ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44b8eac2-d0db-4d4a-bf92-f8831dae06b9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56632025-04-30T06:13:18.144ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
56642025-04-30T06:13:18.144ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9800283-612a-43ea-8ca4-920598c2db3f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56652025-04-30T06:13:18.144ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
56662025-04-30T06:13:18.146ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f2669e2-cd03-4434-9d79-31ba7c48e618", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56672025-04-30T06:13:18.146ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
56682025-04-30T06:13:18.150ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "874f9dd7-7550-4ef7-b650-e8022cd132b1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56692025-04-30T06:13:18.150ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
56702025-04-30T06:13:18.156ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
56712025-04-30T06:13:18.156ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc721b6c-45df-4eb9-a056-5da058f38d50", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56722025-04-30T06:13:18.156ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
56732025-04-30T06:13:18.166ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2520521-01a9-41ef-8adb-2a7b97b218c8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56742025-04-30T06:13:18.168ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
56752025-04-30T06:13:18.171ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fb94228-5524-4167-a3c4-de7931333606", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56762025-04-30T06:13:18.171ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
56772025-04-30T06:13:18.178ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d118369b-cfb8-4f62-9d06-71dc81b0cfaf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56782025-04-30T06:13:18.178ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
56792025-04-30T06:13:18.180ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e158d51-d0d9-466b-a565-12c06362c241", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56802025-04-30T06:13:18.181ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
56812025-04-30T06:13:18.182ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ee279ba-167f-4a58-87b9-41b553570b78", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56822025-04-30T06:13:18.182ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
56832025-04-30T06:13:18.184ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8ebfcf2-c1f5-4695-a8b3-0917fdc31c33", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56842025-04-30T06:13:18.184ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
56852025-04-30T06:13:18.186ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77c16222-942b-4be2-87b9-aff7e759a5d8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56862025-04-30T06:13:18.186ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
56872025-04-30T06:13:18.188ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "acc41def-f3a7-4e5f-9a78-dad60a048307", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56882025-04-30T06:13:18.188ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
56892025-04-30T06:13:18.190ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "953ec4b0-99d4-49ac-963d-4494d4cfc9e6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56902025-04-30T06:13:18.190ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
56912025-04-30T06:13:18.193ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f742732e-b09d-4d66-8e46-ed40df530fc0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56922025-04-30T06:13:18.193ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
56932025-04-30T06:13:18.194ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
56942025-04-30T06:13:18.196ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98f8a2c4-eafd-4120-9efb-332f4fbd0835", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56952025-04-30T06:13:18.196ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
56962025-04-30T06:13:18.198ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "01556fca-4aa3-4ee3-ab9e-001fd100bf9e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56972025-04-30T06:13:18.199ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
56982025-04-30T06:13:18.201ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a203d93-6af2-44d6-b283-96832b236815", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
56992025-04-30T06:13:18.201ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
57002025-04-30T06:13:18.204ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35ab08b7-33c7-4606-972b-0602e09970f8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57012025-04-30T06:13:18.204ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
57022025-04-30T06:13:18.209ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
57032025-04-30T06:13:18.209ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57042025-04-30T06:13:18.209ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57052025-04-30T06:13:18.209ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
57062025-04-30T06:13:18.210ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57072025-04-30T06:13:18.217ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57082025-04-30T06:13:18.224ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
57092025-04-30T06:13:18.224ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57102025-04-30T06:13:18.224ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
57112025-04-30T06:13:18.224ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57122025-04-30T06:13:18.225ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "56342566-8a93-482a-a7b6-a21f2b3b00b7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57132025-04-30T06:13:18.225ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
57142025-04-30T06:13:18.227ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f17368d1-ad70-4e78-b0d7-a5d022e8220c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57152025-04-30T06:13:18.227ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57162025-04-30T06:13:18.227ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
57172025-04-30T06:13:18.227ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
57182025-04-30T06:13:18.228ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57192025-04-30T06:13:18.229ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "854a5228-461f-4da5-962a-5d44483ba6e6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57202025-04-30T06:13:18.229ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
57212025-04-30T06:13:18.230ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "379fc553-cb33-4f1b-9bb8-d88bb78ca5e4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57222025-04-30T06:13:18.231ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
57232025-04-30T06:13:18.232ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0882378-03b7-4cda-ae84-e7f53c269a16", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57242025-04-30T06:13:18.232ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
57252025-04-30T06:13:18.234ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c17ef3b-dcce-46b8-97e6-a799d330a94c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57262025-04-30T06:13:18.235ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
57272025-04-30T06:13:18.237ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b989da9a-af13-47a0-b940-0d39d6f20a3c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57282025-04-30T06:13:18.238ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
57292025-04-30T06:13:18.240ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "85eccbc8-45e1-4f71-a707-150a3b67a86d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57302025-04-30T06:13:18.240ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
57312025-04-30T06:13:18.243ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "511dc35c-bbdb-4e54-9a5e-82654a6c0564", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57322025-04-30T06:13:18.243ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
57332025-04-30T06:13:18.247ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "53ed7e06-16e9-4afb-b502-99b8b51f44d6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57342025-04-30T06:13:18.248ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
57352025-04-30T06:13:18.250ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f70874fc-40b5-4cc2-8cfa-4176b53b34ea", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57362025-04-30T06:13:18.250ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
57372025-04-30T06:13:18.252ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9664e8d-67d3-4623-aeda-28f1d5d37b6d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57382025-04-30T06:13:18.252ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
57392025-04-30T06:13:18.253ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "78d3d8c2-b76a-43bf-a221-e9d0a48d16d3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57402025-04-30T06:13:18.254ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
57412025-04-30T06:13:18.255ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "36b751b2-d42f-42d7-b278-e6d42ee34d6c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57422025-04-30T06:13:18.255ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
57432025-04-30T06:13:18.256ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
57442025-04-30T06:13:18.257ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9555c725-bb28-40b3-865f-e03a22dc8458", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
57452025-04-30T06:13:18.257ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
57462025-04-30T06:13:18.259ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa9ce960-143e-4d90-81fe-00e596b20ff4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
57472025-04-30T06:13:18.259ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
57482025-04-30T06:13:18.259ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57492025-04-30T06:13:18.259ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57502025-04-30T06:13:18.259ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57512025-04-30T06:13:18.260ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57522025-04-30T06:13:18.262ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57532025-04-30T06:13:18.266ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
57542025-04-30T06:13:18.267ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
57552025-04-30T06:13:18.267ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57562025-04-30T06:13:18.267ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57572025-04-30T06:13:18.267ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57582025-04-30T06:13:18.268ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57592025-04-30T06:13:18.272ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57602025-04-30T06:13:18.276ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
57612025-04-30T06:13:18.276ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57622025-04-30T06:13:18.276ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57632025-04-30T06:13:18.277ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57642025-04-30T06:13:18.307ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
57652025-04-30T06:13:18.307ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57662025-04-30T06:13:18.307ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57672025-04-30T06:13:18.308ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
57682025-04-30T06:13:18.308ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57692025-04-30T06:13:18.325ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
57702025-04-30T06:13:18.325ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
57712025-04-30T06:13:18.325ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:41068 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
57722025-04-30T06:13:18.326ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:41068 req_id = 53f6a6e7-83ca-457a-a3d1-3d76db48e2eb sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
57732025-04-30T06:13:18.326ZINFOomicron-dev (dropshot): request completed latency_us = 517 local_addr = [::1]:61881 method = GET remote_addr = [::1]:41068 req_id = 53f6a6e7-83ca-457a-a3d1-3d76db48e2eb response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
57742025-04-30T06:13:18.327ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "53f6a6e7-83ca-457a-a3d1-3d76db48e2eb", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
57752025-04-30T06:13:18.328ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57762025-04-30T06:13:18.329ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
57772025-04-30T06:13:18.329ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
57782025-04-30T06:13:18.329ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
57792025-04-30T06:13:18.331ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
57802025-04-30T06:13:18.331ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
57812025-04-30T06:13:18.331ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57822025-04-30T06:13:18.331ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57832025-04-30T06:13:18.332ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57842025-04-30T06:13:18.335ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
57852025-04-30T06:13:18.335ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57862025-04-30T06:13:18.335ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
57872025-04-30T06:13:18.335ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57882025-04-30T06:13:18.336ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
57892025-04-30T06:13:18.338ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
57902025-04-30T06:13:18.338ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
57912025-04-30T06:13:18.339ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
57922025-04-30T06:13:18.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
57932025-04-30T06:13:18.341ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
57942025-04-30T06:13:18.341ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
57952025-04-30T06:13:18.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
57962025-04-30T06:13:18.372ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
57972025-04-30T06:13:18.372ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
57982025-04-30T06:13:18.373ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 325.550949ms iteration = 15 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
57992025-04-30T06:13:18.373ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 16 reason = Signaled
58002025-04-30T06:13:18.373ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
58012025-04-30T06:13:18.373ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
58022025-04-30T06:13:18.373ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
58032025-04-30T06:13:18.373ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
58042025-04-30T06:13:18.375ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
58052025-04-30T06:13:18.377ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58062025-04-30T06:13:18.385ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
58072025-04-30T06:13:18.385ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
58082025-04-30T06:13:18.385ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
58092025-04-30T06:13:18.386ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
58102025-04-30T06:13:18.390ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
58112025-04-30T06:13:18.391ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
58122025-04-30T06:13:18.391ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 53 e1 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 S............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52651 req_id = 49596f51-9eca-4c00-bf48-1b313859cdbb
58132025-04-30T06:13:18.392ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 21473,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52651 req_id = 49596f51-9eca-4c00-bf48-1b313859cdbb
58142025-04-30T06:13:18.393ZDEBGomicron-dev (store): query key key = _mgs._tcp
58152025-04-30T06:13:18.393ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
58162025-04-30T06:13:18.393ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52651 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 49596f51-9eca-4c00-bf48-1b313859cdbb
58172025-04-30T06:13:18.393ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2463, tv_nsec: 191930066 } })
58182025-04-30T06:13:18.393ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
58192025-04-30T06:13:18.394ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 63 07 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e c...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60117 req_id = 7ef05dbc-97bc-47ae-8865-a6e05cee649e
58202025-04-30T06:13:18.394ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25351,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60117 req_id = 7ef05dbc-97bc-47ae-8865-a6e05cee649e
58212025-04-30T06:13:18.394ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
58222025-04-30T06:13:18.394ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60117 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7ef05dbc-97bc-47ae-8865-a6e05cee649e
58232025-04-30T06:13:18.408ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
58242025-04-30T06:13:18.431ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
58252025-04-30T06:13:18.432ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 58 03 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c X............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:49851 req_id = 46392128-cd39-4e50-b127-3b15f652d512
58262025-04-30T06:13:18.432ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22531,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49851 req_id = 46392128-cd39-4e50-b127-3b15f652d512
58272025-04-30T06:13:18.433ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
58282025-04-30T06:13:18.433ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22531, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49851 req_id = 46392128-cd39-4e50-b127-3b15f652d512
58292025-04-30T06:13:18.433ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
58302025-04-30T06:13:18.434ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
58312025-04-30T06:13:18.434ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
58322025-04-30T06:13:18.438ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9de296ed-4ec0-43f2-91c7-563673e1eedd", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58332025-04-30T06:13:18.438ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
58342025-04-30T06:13:18.443ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "48daee25-8854-4c56-b501-482ff080e0d2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58352025-04-30T06:13:18.443ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
58362025-04-30T06:13:18.446ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c137c72-551a-46d6-9310-07166f25657d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58372025-04-30T06:13:18.446ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
58382025-04-30T06:13:18.448ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b54e3a30-b71b-4983-8f39-cfcc011cd590", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58392025-04-30T06:13:18.448ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
58402025-04-30T06:13:18.451ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7bc19144-9667-4c74-a58d-47f3bf5aeee4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58412025-04-30T06:13:18.451ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
58422025-04-30T06:13:18.454ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "78cd7bc4-8439-4b2d-857e-e7ded4a6be27", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58432025-04-30T06:13:18.454ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
58442025-04-30T06:13:18.459ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1287fde1-1a06-48fc-9980-19d880071b0d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58452025-04-30T06:13:18.459ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
58462025-04-30T06:13:18.459ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
58472025-04-30T06:13:18.459ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
58482025-04-30T06:13:18.459ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
58492025-04-30T06:13:18.459ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
58502025-04-30T06:13:18.461ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
58512025-04-30T06:13:18.463ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4014ad57-307b-4bce-af56-848fe852935b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58522025-04-30T06:13:18.463ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
58532025-04-30T06:13:18.465ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0582c179-c500-4a58-a5ef-3756bb3b32b2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58542025-04-30T06:13:18.465ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
58552025-04-30T06:13:18.466ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1af8e3dd-786f-4371-b462-aadee4056151", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58562025-04-30T06:13:18.467ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
58572025-04-30T06:13:18.468ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
58582025-04-30T06:13:18.469ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "65983ab4-1af2-4b88-a78e-0e159b9e9f5a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58592025-04-30T06:13:18.470ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
58602025-04-30T06:13:18.471ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c550270-038c-4fa9-89b2-2400cad3d27c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58612025-04-30T06:13:18.471ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
58622025-04-30T06:13:18.473ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "32b97161-7ce4-4ebe-b77c-79ba82cd87d9", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58632025-04-30T06:13:18.473ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
58642025-04-30T06:13:18.479ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eae1e50f-8c7e-4af9-8cf0-ecace1209970", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58652025-04-30T06:13:18.479ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
58662025-04-30T06:13:18.480ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
58672025-04-30T06:13:18.480ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
58682025-04-30T06:13:18.480ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
58692025-04-30T06:13:18.481ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
58702025-04-30T06:13:18.481ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "22d60b48-66d4-460e-a815-fcb6fa92c575", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58712025-04-30T06:13:18.481ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
58722025-04-30T06:13:18.484ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd6f11b7-8a40-40cc-a8f1-b65db3f6596e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58732025-04-30T06:13:18.484ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
58742025-04-30T06:13:18.484ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
58752025-04-30T06:13:18.485ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
58762025-04-30T06:13:18.486ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c944e57a-9526-4406-aab2-1856b5330fae", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58772025-04-30T06:13:18.486ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
58782025-04-30T06:13:18.486ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
58792025-04-30T06:13:18.488ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc89738d-f456-409d-befe-6be5639c40e7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58802025-04-30T06:13:18.489ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
58812025-04-30T06:13:18.492ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7e777bc-2ecb-4c8a-99c9-4df45a4a5c9f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58822025-04-30T06:13:18.492ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
58832025-04-30T06:13:18.493ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd75b087-61bc-46a6-86d8-fb8604a1249c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58842025-04-30T06:13:18.493ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
58852025-04-30T06:13:18.502ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "edeec7b5-4ce3-4a7e-98b0-99b866f4e656", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58862025-04-30T06:13:18.502ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
58872025-04-30T06:13:18.502ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af4c45fa-4227-4be5-9044-3da5918c0113", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
58882025-04-30T06:13:18.503ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
58892025-04-30T06:13:18.506ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e23b062-4685-4dd5-9194-8357f8d003a6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58902025-04-30T06:13:18.506ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
58912025-04-30T06:13:18.509ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b3aab29-f597-4e73-929c-f87f5acf3de5", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58922025-04-30T06:13:18.509ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
58932025-04-30T06:13:18.511ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16f52c02-c43f-4129-9871-b0d53ce2027c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58942025-04-30T06:13:18.512ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
58952025-04-30T06:13:18.514ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "270224c1-c572-4674-aa5e-c1e332a38c9b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58962025-04-30T06:13:18.514ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
58972025-04-30T06:13:18.516ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8410e429-83cc-45be-8135-8e0cc087405e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
58982025-04-30T06:13:18.516ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
58992025-04-30T06:13:18.518ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2cc4fb3-8ed8-4177-b6d1-92945e3e2c3b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59002025-04-30T06:13:18.518ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
59012025-04-30T06:13:18.520ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7f18508-718d-4517-a643-ffbe4d29f302", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59022025-04-30T06:13:18.520ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
59032025-04-30T06:13:18.522ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "06c0bbc9-f12b-420d-ba94-3eb569285781", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59042025-04-30T06:13:18.523ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
59052025-04-30T06:13:18.523ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f95ca5b-ad43-4ce8-9a20-83ac91dd2201", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59062025-04-30T06:13:18.524ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
59072025-04-30T06:13:18.525ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "815fa1d7-6d65-4128-8028-3d8e8b0083ae", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59082025-04-30T06:13:18.525ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
59092025-04-30T06:13:18.526ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bbcbd06-f8fa-482c-b960-8dbb39730627", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59102025-04-30T06:13:18.527ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
59112025-04-30T06:13:18.528ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
59122025-04-30T06:13:18.528ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d5ea2ec-ca0b-46c2-9308-f63b2f1f6f68", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59132025-04-30T06:13:18.528ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
59142025-04-30T06:13:18.530ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "df9dbb39-75a1-472a-98df-cee38f463269", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59152025-04-30T06:13:18.530ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
59162025-04-30T06:13:18.532ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bb1ba63-a491-4d7e-b3b4-6e82db01c600", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59172025-04-30T06:13:18.532ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
59182025-04-30T06:13:18.534ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1182d84-b138-4b26-af03-716d838fcf61", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59192025-04-30T06:13:18.535ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
59202025-04-30T06:13:18.537ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92859d8b-4710-41d2-a1f3-0bd39d258ba3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59212025-04-30T06:13:18.537ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
59222025-04-30T06:13:18.539ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae3675d3-d8f5-4e20-86b4-41215bb8929a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59232025-04-30T06:13:18.539ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
59242025-04-30T06:13:18.541ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c1fd491-e155-4cc9-aa8c-dbc70c5d420d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59252025-04-30T06:13:18.541ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
59262025-04-30T06:13:18.543ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "42665895-549c-4b68-87fc-ad0779aa4550", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59272025-04-30T06:13:18.543ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
59282025-04-30T06:13:18.544ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cf9e367-63d9-4262-8f34-ba956246182e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59292025-04-30T06:13:18.544ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
59302025-04-30T06:13:18.545ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "15e15186-26cd-48fa-96f3-5fed9b054e2e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59312025-04-30T06:13:18.545ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
59322025-04-30T06:13:18.547ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f4122e8-f6c4-4307-8b3f-322ef1899167", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59332025-04-30T06:13:18.547ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
59342025-04-30T06:13:18.549ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0ebc5d3-73a6-4c46-abbf-33d9de13d37a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
59352025-04-30T06:13:18.549ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
59362025-04-30T06:13:18.549ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59372025-04-30T06:13:18.549ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59382025-04-30T06:13:18.549ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59392025-04-30T06:13:18.550ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59402025-04-30T06:13:18.554ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59412025-04-30T06:13:18.559ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
59422025-04-30T06:13:18.559ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
59432025-04-30T06:13:18.559ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59442025-04-30T06:13:18.559ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59452025-04-30T06:13:18.559ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59462025-04-30T06:13:18.560ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59472025-04-30T06:13:18.562ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59482025-04-30T06:13:18.567ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
59492025-04-30T06:13:18.567ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59502025-04-30T06:13:18.567ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59512025-04-30T06:13:18.568ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59522025-04-30T06:13:18.580ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
59532025-04-30T06:13:18.580ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59542025-04-30T06:13:18.580ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59552025-04-30T06:13:18.580ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
59562025-04-30T06:13:18.581ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59572025-04-30T06:13:18.634ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
59582025-04-30T06:13:18.635ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
59592025-04-30T06:13:18.635ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:39738 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
59602025-04-30T06:13:18.642ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:39738 req_id = 65805934-4c4f-4ac0-aaf3-fe817bbada0d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
59612025-04-30T06:13:18.642ZINFOomicron-dev (dropshot): request completed latency_us = 5777 local_addr = [::1]:61881 method = GET remote_addr = [::1]:39738 req_id = 65805934-4c4f-4ac0-aaf3-fe817bbada0d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
59622025-04-30T06:13:18.643ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "65805934-4c4f-4ac0-aaf3-fe817bbada0d", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:17 GMT"} })
59632025-04-30T06:13:18.646ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
59642025-04-30T06:13:18.646ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
59652025-04-30T06:13:18.646ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
59662025-04-30T06:13:18.646ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59672025-04-30T06:13:18.649ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
59682025-04-30T06:13:18.649ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
59692025-04-30T06:13:18.649ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59702025-04-30T06:13:18.649ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59712025-04-30T06:13:18.650ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59722025-04-30T06:13:18.654ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
59732025-04-30T06:13:18.654ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
59742025-04-30T06:13:18.654ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59752025-04-30T06:13:18.654ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
59762025-04-30T06:13:18.666ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59772025-04-30T06:13:18.668ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
59782025-04-30T06:13:18.668ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
59792025-04-30T06:13:18.668ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
59802025-04-30T06:13:18.669ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
59812025-04-30T06:13:18.676ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
59822025-04-30T06:13:18.676ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
59832025-04-30T06:13:18.677ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
59842025-04-30T06:13:18.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
59852025-04-30T06:13:18.679ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
59862025-04-30T06:13:18.680ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = e795f0a2-20ab-4f35-bad1-a2a1215fdae3 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
59872025-04-30T06:13:18.681ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = e795f0a2-20ab-4f35-bad1-a2a1215fdae3 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
59882025-04-30T06:13:18.681ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = e795f0a2-20ab-4f35-bad1-a2a1215fdae3 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
59892025-04-30T06:13:18.682ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = e795f0a2-20ab-4f35-bad1-a2a1215fdae3 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
59902025-04-30T06:13:18.693ZINFOomicron-dev (dropshot_internal): request completed latency_us = 13636 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = e795f0a2-20ab-4f35-bad1-a2a1215fdae3 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
59912025-04-30T06:13:18.694ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "e795f0a2-20ab-4f35-bad1-a2a1215fdae3", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59922025-04-30T06:13:18.694ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
59932025-04-30T06:13:18.694ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9405dcb5-20ac-45e1-a84f-2d117e6bc2cd uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
59942025-04-30T06:13:18.695ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9405dcb5-20ac-45e1-a84f-2d117e6bc2cd resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
59952025-04-30T06:13:18.695ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9405dcb5-20ac-45e1-a84f-2d117e6bc2cd roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
59962025-04-30T06:13:18.696ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9405dcb5-20ac-45e1-a84f-2d117e6bc2cd resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
59972025-04-30T06:13:18.703ZINFOomicron-dev (dropshot_internal): request completed latency_us = 8912 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9405dcb5-20ac-45e1-a84f-2d117e6bc2cd response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
59982025-04-30T06:13:18.704ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "9405dcb5-20ac-45e1-a84f-2d117e6bc2cd", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
59992025-04-30T06:13:18.704ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60002025-04-30T06:13:18.704ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
60012025-04-30T06:13:18.704ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
60022025-04-30T06:13:18.712ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
60032025-04-30T06:13:18.713ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
60042025-04-30T06:13:18.713ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 340.138754ms iteration = 16 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
60052025-04-30T06:13:18.713ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 17 reason = Signaled
60062025-04-30T06:13:18.713ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
60072025-04-30T06:13:18.713ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
60082025-04-30T06:13:18.713ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
60092025-04-30T06:13:18.713ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
60102025-04-30T06:13:18.715ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
60112025-04-30T06:13:18.717ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60122025-04-30T06:13:18.721ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60132025-04-30T06:13:18.721ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60142025-04-30T06:13:18.725ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60152025-04-30T06:13:18.725ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60162025-04-30T06:13:18.725ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60172025-04-30T06:13:18.725ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
60182025-04-30T06:13:18.735ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
60192025-04-30T06:13:18.735ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
60202025-04-30T06:13:18.735ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
60212025-04-30T06:13:18.736ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
60222025-04-30T06:13:18.746ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
60232025-04-30T06:13:18.746ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
60242025-04-30T06:13:18.746ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
60252025-04-30T06:13:18.748ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: bf 96 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64279 req_id = 15fa88ae-bdaf-4dac-9b64-99ff91844548
60262025-04-30T06:13:18.749ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49046,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64279 req_id = 15fa88ae-bdaf-4dac-9b64-99ff91844548
60272025-04-30T06:13:18.749ZDEBGomicron-dev (store): query key key = _mgs._tcp
60282025-04-30T06:13:18.750ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
60292025-04-30T06:13:18.750ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64279 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 15fa88ae-bdaf-4dac-9b64-99ff91844548
60302025-04-30T06:13:18.751ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2463, tv_nsec: 550260987 } })
60312025-04-30T06:13:18.751ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
60322025-04-30T06:13:18.752ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 01 24 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .$..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64294 req_id = 3d9a859b-22ed-4059-8bd5-085017e4853b
60332025-04-30T06:13:18.752ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 292,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64294 req_id = 3d9a859b-22ed-4059-8bd5-085017e4853b
60342025-04-30T06:13:18.752ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
60352025-04-30T06:13:18.752ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64294 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3d9a859b-22ed-4059-8bd5-085017e4853b
60362025-04-30T06:13:18.809ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
60372025-04-30T06:13:18.809ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
60382025-04-30T06:13:18.809ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
60392025-04-30T06:13:18.809ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 0c 3b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .;..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52008 req_id = 9570092b-86b1-416a-93ff-34ca533359bb
60402025-04-30T06:13:18.809ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
60412025-04-30T06:13:18.809ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
60422025-04-30T06:13:18.810ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 3131,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52008 req_id = 9570092b-86b1-416a-93ff-34ca533359bb
60432025-04-30T06:13:18.810ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
60442025-04-30T06:13:18.810ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 3131, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52008 req_id = 9570092b-86b1-416a-93ff-34ca533359bb
60452025-04-30T06:13:18.810ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
60462025-04-30T06:13:18.810ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
60472025-04-30T06:13:18.810ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
60482025-04-30T06:13:18.810ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
60492025-04-30T06:13:18.813ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "554a2b32-712a-4b9c-b2d2-9ff55235c38b", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60502025-04-30T06:13:18.813ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
60512025-04-30T06:13:18.816ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
60522025-04-30T06:13:18.818ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3251e1f7-020d-430a-9627-e153b5ea3e53", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60532025-04-30T06:13:18.818ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
60542025-04-30T06:13:18.821ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6507e45e-f538-4d59-b88e-b9d57ec5724a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60552025-04-30T06:13:18.821ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
60562025-04-30T06:13:18.823ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbf1b7c5-ae25-4be4-bb2f-8f2a55d0d321", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60572025-04-30T06:13:18.824ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
60582025-04-30T06:13:18.824ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
60592025-04-30T06:13:18.824ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
60602025-04-30T06:13:18.824ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
60612025-04-30T06:13:18.824ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
60622025-04-30T06:13:18.827ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78a55af3-50f2-4c8f-acec-483801f3839a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60632025-04-30T06:13:18.827ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
60642025-04-30T06:13:18.827ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
60652025-04-30T06:13:18.827ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
60662025-04-30T06:13:18.829ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
60672025-04-30T06:13:18.831ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "77a831e5-b573-410e-9b1c-7aab8b9a48ee", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60682025-04-30T06:13:18.831ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
60692025-04-30T06:13:18.834ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f20c983c-7153-4a06-9e44-63757de602b4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60702025-04-30T06:13:18.834ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
60712025-04-30T06:13:18.838ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "13812f01-71ee-41f9-b4df-a4bacf5b6d79", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60722025-04-30T06:13:18.838ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
60732025-04-30T06:13:18.839ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66d5369d-69b8-4c39-a211-a695d5c2b85c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60742025-04-30T06:13:18.840ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
60752025-04-30T06:13:18.842ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c919f094-6035-474d-9390-e1041c287629", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60762025-04-30T06:13:18.843ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
60772025-04-30T06:13:18.844ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fdc02bc-1d8b-4a02-9f9c-d82d44e6e6f9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60782025-04-30T06:13:18.844ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
60792025-04-30T06:13:18.847ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "17fc6841-3d61-4857-b3e0-e006adf4c6fe", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60802025-04-30T06:13:18.847ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
60812025-04-30T06:13:18.850ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76c14021-d344-42ac-b03d-47bd2ded2df8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60822025-04-30T06:13:18.852ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
60832025-04-30T06:13:18.859ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9edd5532-0eed-47b1-9232-ccac1977d7c7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60842025-04-30T06:13:18.860ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
60852025-04-30T06:13:18.868ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f1943fa-dd55-46c3-ac49-d528554dd4dd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60862025-04-30T06:13:18.868ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
60872025-04-30T06:13:18.873ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2156fb3e-a422-4556-9ba1-5f0435c5b68e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60882025-04-30T06:13:18.873ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
60892025-04-30T06:13:18.880ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "17e9656f-fe19-4f77-b22e-d760775ffc79", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60902025-04-30T06:13:18.880ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
60912025-04-30T06:13:18.883ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfcb6c78-435e-4125-b41e-bba3a5e35b99", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60922025-04-30T06:13:18.883ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
60932025-04-30T06:13:18.886ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d5530f8-53ac-4b9c-ad2d-7c8a967e51fe", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60942025-04-30T06:13:18.886ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
60952025-04-30T06:13:18.887ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b926336-dfc4-4438-8832-e9a9fba216bf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60962025-04-30T06:13:18.887ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
60972025-04-30T06:13:18.887ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
60982025-04-30T06:13:18.889ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "87932606-27c9-4a97-89db-b7f7a9f92dba", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
60992025-04-30T06:13:18.889ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
61002025-04-30T06:13:18.890ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e97e182-2ba6-4a53-b5f3-60b623af43f3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61012025-04-30T06:13:18.890ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
61022025-04-30T06:13:18.892ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd7738be-43b2-468a-9df2-18382eea5803", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61032025-04-30T06:13:18.892ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
61042025-04-30T06:13:18.892ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "08bc9a7b-a1d2-4d06-bfa2-692f07c2a960", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61052025-04-30T06:13:18.892ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
61062025-04-30T06:13:18.897ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef670df4-7c99-44af-b744-f9adc52fe3b2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61072025-04-30T06:13:18.897ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
61082025-04-30T06:13:18.901ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2a59acc-a8a9-4189-bf8f-5a349f2afc1b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61092025-04-30T06:13:18.902ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
61102025-04-30T06:13:18.907ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e46f3fe3-8c06-4fd4-8a20-52cc44c16731", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61112025-04-30T06:13:18.909ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
61122025-04-30T06:13:18.913ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4618286c-1d7a-4887-a23a-098135b2b92b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61132025-04-30T06:13:18.913ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
61142025-04-30T06:13:18.914ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4c29d09-af80-49f0-bac2-9d6f3213491b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61152025-04-30T06:13:18.915ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
61162025-04-30T06:13:18.926ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cb88e06-a431-4098-95d4-7f431555cd76", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61172025-04-30T06:13:18.941ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
61182025-04-30T06:13:18.943ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9782d01-935f-4642-b479-095bccc41cf4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61192025-04-30T06:13:18.943ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
61202025-04-30T06:13:18.944ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a66f299-403c-4061-9c75-1bfd0b720305", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61212025-04-30T06:13:18.944ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
61222025-04-30T06:13:18.945ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5208eef-fba0-4f29-aaee-b380ba1931d1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61232025-04-30T06:13:18.945ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
61242025-04-30T06:13:18.946ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60d281a7-1ec5-45e5-b462-0377bc2c374d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61252025-04-30T06:13:18.946ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
61262025-04-30T06:13:18.947ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6e6cf1b-e0d4-4e46-a7c9-9bb2c60e371d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61272025-04-30T06:13:18.948ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
61282025-04-30T06:13:18.949ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
61292025-04-30T06:13:18.949ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61302025-04-30T06:13:18.949ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61312025-04-30T06:13:18.949ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
61322025-04-30T06:13:18.949ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe40d606-c919-4a74-85a6-03f4eb0f6957", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61332025-04-30T06:13:18.949ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
61342025-04-30T06:13:18.950ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61352025-04-30T06:13:18.953ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "32b14307-627d-4ffc-ba30-5c99e157e113", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61362025-04-30T06:13:18.953ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
61372025-04-30T06:13:18.956ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a60c8fa-88d9-4ed1-abce-7506c0abcc1d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61382025-04-30T06:13:18.956ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
61392025-04-30T06:13:18.961ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c39fa448-8745-4c24-9558-16efcc88b267", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61402025-04-30T06:13:18.961ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
61412025-04-30T06:13:18.969ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4942c862-4d3f-44cd-ab91-a661bc4bd95e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61422025-04-30T06:13:18.970ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
61432025-04-30T06:13:18.970ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61442025-04-30T06:13:18.978ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "faf6b165-e90e-446f-b370-25b3daffda51", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61452025-04-30T06:13:18.978ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
61462025-04-30T06:13:18.981ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e519ce9-4cc8-4603-9ef9-8699f97a490a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61472025-04-30T06:13:18.981ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
61482025-04-30T06:13:18.983ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8b00a36-f51f-4985-ba44-30e852d20b50", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61492025-04-30T06:13:18.983ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
61502025-04-30T06:13:18.983ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61512025-04-30T06:13:18.983ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
61522025-04-30T06:13:18.983ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61532025-04-30T06:13:18.989ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
61542025-04-30T06:13:18.991ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f316c1e-03aa-4d04-b6ad-d9bda0d8e26d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61552025-04-30T06:13:18.991ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
61562025-04-30T06:13:18.992ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1a11338-29cf-43b0-a39f-a7af0e9f2612", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61572025-04-30T06:13:18.992ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
61582025-04-30T06:13:18.992ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61592025-04-30T06:13:18.992ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61602025-04-30T06:13:18.993ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61612025-04-30T06:13:18.993ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61622025-04-30T06:13:18.994ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
61632025-04-30T06:13:18.995ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
61642025-04-30T06:13:18.995ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
61652025-04-30T06:13:18.996ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61662025-04-30T06:13:19.001ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61672025-04-30T06:13:19.001ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
61682025-04-30T06:13:19.001ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61692025-04-30T06:13:19.001ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61702025-04-30T06:13:19.001ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61712025-04-30T06:13:19.001ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61722025-04-30T06:13:19.004ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
61732025-04-30T06:13:19.017ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
61742025-04-30T06:13:19.018ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61752025-04-30T06:13:19.018ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61762025-04-30T06:13:19.018ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
61772025-04-30T06:13:19.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
61782025-04-30T06:13:19.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
61792025-04-30T06:13:19.091ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
61802025-04-30T06:13:19.091ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
61812025-04-30T06:13:19.095ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
61822025-04-30T06:13:19.095ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
61832025-04-30T06:13:19.098ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
61842025-04-30T06:13:19.098ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
61852025-04-30T06:13:19.102ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
61862025-04-30T06:13:19.142ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
61872025-04-30T06:13:19.142ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
61882025-04-30T06:13:19.143ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:57955 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
61892025-04-30T06:13:19.144ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:57955 req_id = b9aee5bf-f9a3-4afe-a19a-ebdb462f5f01 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
61902025-04-30T06:13:19.145ZINFOomicron-dev (dropshot): request completed latency_us = 617 local_addr = [::1]:61881 method = GET remote_addr = [::1]:57955 req_id = b9aee5bf-f9a3-4afe-a19a-ebdb462f5f01 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
61912025-04-30T06:13:19.145ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9aee5bf-f9a3-4afe-a19a-ebdb462f5f01", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
61922025-04-30T06:13:19.148ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61932025-04-30T06:13:19.148ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
61942025-04-30T06:13:19.148ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
61952025-04-30T06:13:19.150ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
61962025-04-30T06:13:19.150ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
61972025-04-30T06:13:19.150ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
61982025-04-30T06:13:19.150ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
61992025-04-30T06:13:19.151ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
62002025-04-30T06:13:19.157ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62012025-04-30T06:13:19.164ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
62022025-04-30T06:13:19.164ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
62032025-04-30T06:13:19.164ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62042025-04-30T06:13:19.164ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
62052025-04-30T06:13:19.164ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
62062025-04-30T06:13:19.165ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62072025-04-30T06:13:19.165ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
62082025-04-30T06:13:19.165ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
62092025-04-30T06:13:19.165ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
62102025-04-30T06:13:19.169ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62112025-04-30T06:13:19.175ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
62122025-04-30T06:13:19.175ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62132025-04-30T06:13:19.175ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
62142025-04-30T06:13:19.176ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62152025-04-30T06:13:19.178ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62162025-04-30T06:13:19.178ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
62172025-04-30T06:13:19.179ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62182025-04-30T06:13:19.198ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
62192025-04-30T06:13:19.203ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
62202025-04-30T06:13:19.204ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
62212025-04-30T06:13:19.204ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 491.135816ms iteration = 17 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
62222025-04-30T06:13:19.204ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 18 reason = Signaled
62232025-04-30T06:13:19.204ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
62242025-04-30T06:13:19.205ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
62252025-04-30T06:13:19.205ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
62262025-04-30T06:13:19.206ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
62272025-04-30T06:13:19.208ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
62282025-04-30T06:13:19.217ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62292025-04-30T06:13:19.221ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
62302025-04-30T06:13:19.222ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
62312025-04-30T06:13:19.222ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
62322025-04-30T06:13:19.222ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
62332025-04-30T06:13:19.223ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
62342025-04-30T06:13:19.226ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
62352025-04-30T06:13:19.227ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
62362025-04-30T06:13:19.228ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b7 ce 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:57256 req_id = 10a699aa-0e73-45c1-aa71-daf1f05d0640
62372025-04-30T06:13:19.228ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 47054,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57256 req_id = 10a699aa-0e73-45c1-aa71-daf1f05d0640
62382025-04-30T06:13:19.228ZDEBGomicron-dev (store): query key key = _mgs._tcp
62392025-04-30T06:13:19.228ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
62402025-04-30T06:13:19.228ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57256 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 10a699aa-0e73-45c1-aa71-daf1f05d0640
62412025-04-30T06:13:19.228ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2464, tv_nsec: 27839892 } })
62422025-04-30T06:13:19.229ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
62432025-04-30T06:13:19.229ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 5f 9c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e _...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59973 req_id = ce9124d5-0b4d-45e9-b712-2737e8d7b76f
62442025-04-30T06:13:19.230ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24476,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59973 req_id = ce9124d5-0b4d-45e9-b712-2737e8d7b76f
62452025-04-30T06:13:19.230ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
62462025-04-30T06:13:19.230ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59973 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ce9124d5-0b4d-45e9-b712-2737e8d7b76f
62472025-04-30T06:13:19.273ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
62482025-04-30T06:13:19.273ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
62492025-04-30T06:13:19.273ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
62502025-04-30T06:13:19.273ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62512025-04-30T06:13:19.273ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
62522025-04-30T06:13:19.274ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: e3 4b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .K..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57383 req_id = aa65aebf-4d4e-4679-aede-18728811f77c
62532025-04-30T06:13:19.274ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 58187,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57383 req_id = aa65aebf-4d4e-4679-aede-18728811f77c
62542025-04-30T06:13:19.274ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62552025-04-30T06:13:19.275ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
62562025-04-30T06:13:19.275ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 58187, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57383 req_id = aa65aebf-4d4e-4679-aede-18728811f77c
62572025-04-30T06:13:19.275ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
62582025-04-30T06:13:19.275ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
62592025-04-30T06:13:19.275ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
62602025-04-30T06:13:19.277ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
62612025-04-30T06:13:19.278ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b7cab8a-d086-4215-a4b0-c14def9e0326", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62622025-04-30T06:13:19.279ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
62632025-04-30T06:13:19.280ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0bfe6d9-8c27-4b6e-8e31-2b9137d0808b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62642025-04-30T06:13:19.281ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
62652025-04-30T06:13:19.283ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
62662025-04-30T06:13:19.283ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62672025-04-30T06:13:19.283ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "72d81a32-5c7e-4cdb-a608-ba6584218d29", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62682025-04-30T06:13:19.283ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
62692025-04-30T06:13:19.284ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
62702025-04-30T06:13:19.285ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62712025-04-30T06:13:19.286ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fef721c-f153-4ec4-9345-4feca999fc05", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62722025-04-30T06:13:19.286ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
62732025-04-30T06:13:19.287ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
62742025-04-30T06:13:19.287ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
62752025-04-30T06:13:19.289ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e979b58-788a-42de-873e-f21e7e8304e1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62762025-04-30T06:13:19.289ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
62772025-04-30T06:13:19.290ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
62782025-04-30T06:13:19.293ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad6e665b-6b35-4820-9176-e048ac56fef4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62792025-04-30T06:13:19.294ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
62802025-04-30T06:13:19.297ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bc03e71-a9c0-4a5a-bfde-5ad2295f765d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62812025-04-30T06:13:19.297ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
62822025-04-30T06:13:19.301ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e47afab-3e0b-4da8-a371-86b02ab9d9f7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62832025-04-30T06:13:19.302ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
62842025-04-30T06:13:19.303ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d96984a-55b8-4390-b1e9-bea915c815da", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62852025-04-30T06:13:19.303ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
62862025-04-30T06:13:19.305ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45414bcd-83da-43c7-9bd7-d7cfea7d85af", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62872025-04-30T06:13:19.305ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
62882025-04-30T06:13:19.307ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33a6d107-6e99-4fbb-86d9-b4e109877560", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62892025-04-30T06:13:19.307ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
62902025-04-30T06:13:19.309ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1100961d-2833-4d55-8dd3-d29a20622649", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62912025-04-30T06:13:19.309ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
62922025-04-30T06:13:19.311ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7bfaec0-e33a-4516-bbc9-6b9bad4fbc61", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62932025-04-30T06:13:19.311ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
62942025-04-30T06:13:19.313ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea04f511-2353-4ddb-9fdd-91ce612c1d50", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62952025-04-30T06:13:19.313ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
62962025-04-30T06:13:19.315ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "380ac68c-6ee8-455e-a30c-e5faeaa455ee", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62972025-04-30T06:13:19.315ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
62982025-04-30T06:13:19.317ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bcce82f-3640-4925-8f35-0e23dee7fba9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
62992025-04-30T06:13:19.317ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
63002025-04-30T06:13:19.319ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ff92e65-5d95-4008-9ce7-73c8571db5f2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63012025-04-30T06:13:19.319ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
63022025-04-30T06:13:19.320ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
63032025-04-30T06:13:19.322ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "da8e60fb-fcf1-4e86-9e35-ebebcfdf541d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63042025-04-30T06:13:19.322ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
63052025-04-30T06:13:19.325ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c434399a-8b92-4f03-b458-a1b0ed2c0447", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63062025-04-30T06:13:19.325ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
63072025-04-30T06:13:19.325ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d777b132-609f-48f0-a9a0-2b49ef79bd16", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63082025-04-30T06:13:19.326ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
63092025-04-30T06:13:19.327ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02539bca-9706-4626-9fdb-147ba1bbc327", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63102025-04-30T06:13:19.327ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
63112025-04-30T06:13:19.330ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3bf1bf1-7804-492f-903d-5d72371f0dde", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63122025-04-30T06:13:19.330ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
63132025-04-30T06:13:19.332ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e65398d-2cb5-4094-ab79-da1bafa367e3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63142025-04-30T06:13:19.333ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
63152025-04-30T06:13:19.335ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "67b2176d-1c23-4391-8d4a-69a536f7c10f", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63162025-04-30T06:13:19.335ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
63172025-04-30T06:13:19.338ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "24334891-40ed-43ed-8f50-f9a96c57088d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63182025-04-30T06:13:19.338ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
63192025-04-30T06:13:19.340ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e19f6da-f216-4618-86fb-0bbb39f02f9c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63202025-04-30T06:13:19.340ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
63212025-04-30T06:13:19.342ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "10daf464-abad-4412-bc43-478395887f86", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63222025-04-30T06:13:19.342ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
63232025-04-30T06:13:19.344ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2278aaf7-fad2-4695-ad56-da3684cb9a76", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63242025-04-30T06:13:19.344ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
63252025-04-30T06:13:19.346ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cfb2d41-3ddd-43aa-9bae-c17c3c3ad39b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63262025-04-30T06:13:19.346ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
63272025-04-30T06:13:19.348ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "28366f91-6639-4d56-8998-9791e9a8b255", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63282025-04-30T06:13:19.348ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
63292025-04-30T06:13:19.349ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3613956-ec4f-4cb8-b3a6-251a4656de06", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63302025-04-30T06:13:19.349ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
63312025-04-30T06:13:19.351ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c9f8582-eb3e-439b-8acd-3dfb6f327929", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63322025-04-30T06:13:19.351ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
63332025-04-30T06:13:19.352ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6618bd8-f5c1-47e3-90dc-07f26e43fb99", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63342025-04-30T06:13:19.352ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
63352025-04-30T06:13:19.354ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad34b967-9336-4de7-8e67-9e8c55ed7643", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63362025-04-30T06:13:19.354ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
63372025-04-30T06:13:19.355ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b1f8193-bb8d-45b6-bbea-07d707a61c3d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63382025-04-30T06:13:19.356ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
63392025-04-30T06:13:19.357ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9fc9925-faa7-4792-878b-79017a815a1c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63402025-04-30T06:13:19.358ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
63412025-04-30T06:13:19.360ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80137d7a-c2d8-4d52-989c-031035b96b24", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63422025-04-30T06:13:19.360ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
63432025-04-30T06:13:19.362ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d398f988-fae6-4bab-8bfa-4c8ae6962f79", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63442025-04-30T06:13:19.362ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
63452025-04-30T06:13:19.364ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "201067e7-5365-4c6f-940d-c8d4924c11f8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63462025-04-30T06:13:19.364ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
63472025-04-30T06:13:19.367ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cbc706d-dde4-4ce6-b307-833b2646d3e1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63482025-04-30T06:13:19.367ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
63492025-04-30T06:13:19.369ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b321ea6-b763-4eb8-9e72-eef449c9db97", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63502025-04-30T06:13:19.370ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
63512025-04-30T06:13:19.371ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
63522025-04-30T06:13:19.371ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63532025-04-30T06:13:19.371ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63542025-04-30T06:13:19.371ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
63552025-04-30T06:13:19.371ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "09522c6f-3280-4e3c-a1df-98e517681487", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63562025-04-30T06:13:19.372ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
63572025-04-30T06:13:19.372ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63582025-04-30T06:13:19.373ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "91694c96-31dc-4d51-bae0-402f730de4fd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63592025-04-30T06:13:19.374ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
63602025-04-30T06:13:19.375ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63612025-04-30T06:13:19.377ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fe16ec5-5d02-41dc-b173-ae315278ccf0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63622025-04-30T06:13:19.377ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
63632025-04-30T06:13:19.381ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
63642025-04-30T06:13:19.381ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63652025-04-30T06:13:19.381ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
63662025-04-30T06:13:19.382ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63672025-04-30T06:13:19.384ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "84033221-8bc4-4df8-9fb5-966fde171d21", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63682025-04-30T06:13:19.385ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
63692025-04-30T06:13:19.385ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63702025-04-30T06:13:19.385ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63712025-04-30T06:13:19.385ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63722025-04-30T06:13:19.386ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
63732025-04-30T06:13:19.386ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
63742025-04-30T06:13:19.386ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63752025-04-30T06:13:19.386ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
63762025-04-30T06:13:19.392ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63772025-04-30T06:13:19.398ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
63782025-04-30T06:13:19.398ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
63792025-04-30T06:13:19.398ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
63802025-04-30T06:13:19.398ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63812025-04-30T06:13:19.398ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63822025-04-30T06:13:19.399ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63832025-04-30T06:13:19.421ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
63842025-04-30T06:13:19.435ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
63852025-04-30T06:13:19.435ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
63862025-04-30T06:13:19.435ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
63872025-04-30T06:13:19.436ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
63882025-04-30T06:13:19.460ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
63892025-04-30T06:13:19.505ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
63902025-04-30T06:13:19.505ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
63912025-04-30T06:13:19.505ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:50823 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
63922025-04-30T06:13:19.506ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:50823 req_id = 6da959a0-ce11-477a-ae87-b49960574903 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
63932025-04-30T06:13:19.507ZINFOomicron-dev (dropshot): request completed latency_us = 506 local_addr = [::1]:61881 method = GET remote_addr = [::1]:50823 req_id = 6da959a0-ce11-477a-ae87-b49960574903 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
63942025-04-30T06:13:19.507ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "6da959a0-ce11-477a-ae87-b49960574903", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
63952025-04-30T06:13:19.510ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
63962025-04-30T06:13:19.510ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
63972025-04-30T06:13:19.510ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
63982025-04-30T06:13:19.511ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
63992025-04-30T06:13:19.511ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64002025-04-30T06:13:19.511ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64012025-04-30T06:13:19.511ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
64022025-04-30T06:13:19.512ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
64032025-04-30T06:13:19.512ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64042025-04-30T06:13:19.512ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64052025-04-30T06:13:19.512ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64062025-04-30T06:13:19.513ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64072025-04-30T06:13:19.513ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64082025-04-30T06:13:19.516ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64092025-04-30T06:13:19.517ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64102025-04-30T06:13:19.522ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
64112025-04-30T06:13:19.522ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64122025-04-30T06:13:19.522ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64132025-04-30T06:13:19.522ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
64142025-04-30T06:13:19.522ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64152025-04-30T06:13:19.522ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64162025-04-30T06:13:19.523ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
64172025-04-30T06:13:19.524ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64182025-04-30T06:13:19.526ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64192025-04-30T06:13:19.526ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
64202025-04-30T06:13:19.529ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64212025-04-30T06:13:19.557ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
64222025-04-30T06:13:19.557ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
64232025-04-30T06:13:19.558ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 353.647179ms iteration = 18 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
64242025-04-30T06:13:19.558ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 19 reason = Signaled
64252025-04-30T06:13:19.559ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
64262025-04-30T06:13:19.562ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64272025-04-30T06:13:19.562ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64282025-04-30T06:13:19.562ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64292025-04-30T06:13:19.564ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64302025-04-30T06:13:19.572ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
64312025-04-30T06:13:19.574ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64322025-04-30T06:13:19.582ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
64332025-04-30T06:13:19.582ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
64342025-04-30T06:13:19.582ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
64352025-04-30T06:13:19.582ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
64362025-04-30T06:13:19.587ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
64372025-04-30T06:13:19.587ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
64382025-04-30T06:13:19.589ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d5 83 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:54146 req_id = 8bb21aff-798e-4fff-9403-e75a387f495c
64392025-04-30T06:13:19.589ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54659,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54146 req_id = 8bb21aff-798e-4fff-9403-e75a387f495c
64402025-04-30T06:13:19.589ZDEBGomicron-dev (store): query key key = _mgs._tcp
64412025-04-30T06:13:19.589ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
64422025-04-30T06:13:19.590ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54146 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8bb21aff-798e-4fff-9403-e75a387f495c
64432025-04-30T06:13:19.590ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2464, tv_nsec: 389307002 } })
64442025-04-30T06:13:19.590ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
64452025-04-30T06:13:19.591ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 11 73 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .s..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63679 req_id = 1df4e325-75fd-482b-90f1-9baa3de0c111
64462025-04-30T06:13:19.591ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4467,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63679 req_id = 1df4e325-75fd-482b-90f1-9baa3de0c111
64472025-04-30T06:13:19.592ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
64482025-04-30T06:13:19.592ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63679 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1df4e325-75fd-482b-90f1-9baa3de0c111
64492025-04-30T06:13:19.626ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
64502025-04-30T06:13:19.626ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
64512025-04-30T06:13:19.626ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64522025-04-30T06:13:19.626ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
64532025-04-30T06:13:19.627ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64542025-04-30T06:13:19.630ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
64552025-04-30T06:13:19.635ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
64562025-04-30T06:13:19.635ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64572025-04-30T06:13:19.635ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
64582025-04-30T06:13:19.636ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64592025-04-30T06:13:19.638ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
64602025-04-30T06:13:19.638ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
64612025-04-30T06:13:19.639ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
64622025-04-30T06:13:19.649ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
64632025-04-30T06:13:19.650ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 95 11 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52733 req_id = 9406f7dd-487a-4b2b-b5f8-244edff3db7e
64642025-04-30T06:13:19.650ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38161,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52733 req_id = 9406f7dd-487a-4b2b-b5f8-244edff3db7e
64652025-04-30T06:13:19.650ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
64662025-04-30T06:13:19.650ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38161, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52733 req_id = 9406f7dd-487a-4b2b-b5f8-244edff3db7e
64672025-04-30T06:13:19.651ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
64682025-04-30T06:13:19.651ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
64692025-04-30T06:13:19.651ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
64702025-04-30T06:13:19.654ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bee9da9-2c0d-48b3-9937-aadc5b9acafa", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64712025-04-30T06:13:19.655ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
64722025-04-30T06:13:19.656ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aae0ae76-ff8c-497d-90b4-58b6981e6ccf", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64732025-04-30T06:13:19.656ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
64742025-04-30T06:13:19.661ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "568bf8e5-20a5-4cf2-bf43-61aa76881da6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64752025-04-30T06:13:19.661ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
64762025-04-30T06:13:19.664ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "16b3ee65-b1a5-4835-9404-6d688bf41602", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64772025-04-30T06:13:19.664ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
64782025-04-30T06:13:19.666ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfe458fa-fae2-4fcd-b781-48351a9eaf4f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64792025-04-30T06:13:19.666ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
64802025-04-30T06:13:19.668ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea228d4b-b4bc-499a-beb9-155d0beeb60f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64812025-04-30T06:13:19.669ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
64822025-04-30T06:13:19.669ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
64832025-04-30T06:13:19.671ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c21b6232-c0f9-42eb-8177-7baaffecef35", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64842025-04-30T06:13:19.671ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
64852025-04-30T06:13:19.673ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "be9a23df-69aa-4065-a825-921cc14fb5bd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64862025-04-30T06:13:19.673ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
64872025-04-30T06:13:19.691ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b935d8a4-4513-4834-aa3a-3e212d29f86d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
64882025-04-30T06:13:19.692ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
64892025-04-30T06:13:19.694ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "98ab8d70-bdd7-48ed-8d1f-b4c651af4742", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64902025-04-30T06:13:19.694ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
64912025-04-30T06:13:19.695ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f98ea07-7bd4-4425-a2db-f03bbffd0c34", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64922025-04-30T06:13:19.695ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
64932025-04-30T06:13:19.696ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b11b423-298e-49be-bb2c-460ac1c24b65", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64942025-04-30T06:13:19.697ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
64952025-04-30T06:13:19.698ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5db9392c-1367-4f71-9add-6500cb54e038", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64962025-04-30T06:13:19.698ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
64972025-04-30T06:13:19.700ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1528b8a9-fdf5-4418-a599-d08cc3c748e0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
64982025-04-30T06:13:19.700ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
64992025-04-30T06:13:19.702ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b86136b1-a87d-4ba1-bffc-1a26af9208c6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65002025-04-30T06:13:19.702ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
65012025-04-30T06:13:19.704ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b474d4b-5b8e-4ebd-8b18-d9834b952991", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65022025-04-30T06:13:19.704ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
65032025-04-30T06:13:19.706ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "10ca8e61-148e-4cb0-8451-da2e90eb6a58", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65042025-04-30T06:13:19.706ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
65052025-04-30T06:13:19.710ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "742901e3-3a5c-426a-aba3-f8e0a488dc6f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65062025-04-30T06:13:19.711ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
65072025-04-30T06:13:19.716ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "be64fa3a-d321-4138-bd61-d0997126caa1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65082025-04-30T06:13:19.717ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
65092025-04-30T06:13:19.719ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1529549e-ff24-4797-8a2b-2f3be3981229", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65102025-04-30T06:13:19.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
65112025-04-30T06:13:19.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
65122025-04-30T06:13:19.719ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
65132025-04-30T06:13:19.719ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f164e34-5cb9-4226-a67f-6e69b3f3a074", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65142025-04-30T06:13:19.719ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
65152025-04-30T06:13:19.720ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
65162025-04-30T06:13:19.720ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65172025-04-30T06:13:19.720ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65182025-04-30T06:13:19.720ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
65192025-04-30T06:13:19.720ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eff7c995-d7c6-45e8-9d66-51b5e9e26a94", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65202025-04-30T06:13:19.720ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
65212025-04-30T06:13:19.721ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65222025-04-30T06:13:19.721ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "96e2f066-0cb8-45d2-915b-f09084e0ecf9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65232025-04-30T06:13:19.721ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
65242025-04-30T06:13:19.722ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
65252025-04-30T06:13:19.723ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13977393-f990-4315-b920-f58599dbf456", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65262025-04-30T06:13:19.723ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
65272025-04-30T06:13:19.723ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
65282025-04-30T06:13:19.723ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
65292025-04-30T06:13:19.723ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
65302025-04-30T06:13:19.723ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65312025-04-30T06:13:19.725ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13e22a67-8f07-4498-9e28-b50f4f9b89ed", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65322025-04-30T06:13:19.725ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
65332025-04-30T06:13:19.731ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
65342025-04-30T06:13:19.731ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65352025-04-30T06:13:19.731ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
65362025-04-30T06:13:19.732ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65372025-04-30T06:13:19.736ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2860c860-5165-453d-9998-f9a497d562a3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65382025-04-30T06:13:19.736ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
65392025-04-30T06:13:19.743ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13162400-6225-4511-a882-ce6f2007ea59", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65402025-04-30T06:13:19.743ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
65412025-04-30T06:13:19.744ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
65422025-04-30T06:13:19.744ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
65432025-04-30T06:13:19.746ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
65442025-04-30T06:13:19.752ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "61e4b505-08cc-4c9c-9dae-83c428cded8b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65452025-04-30T06:13:19.752ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
65462025-04-30T06:13:19.755ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ca081b3-09c0-460c-9800-e3034ad4e4fb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65472025-04-30T06:13:19.755ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
65482025-04-30T06:13:19.758ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a478cfc-0a8b-4ada-8073-446e1932b73a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65492025-04-30T06:13:19.758ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
65502025-04-30T06:13:19.759ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6543f76-adca-4193-886f-05d5ef65d9fd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65512025-04-30T06:13:19.759ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
65522025-04-30T06:13:19.760ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a40711b8-8592-4fdf-81e9-ba5e551d9893", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65532025-04-30T06:13:19.760ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
65542025-04-30T06:13:19.761ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f9646b2-e58e-4e17-926a-38661afc5fcb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65552025-04-30T06:13:19.761ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
65562025-04-30T06:13:19.763ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b24c7c3-257c-4fcf-913d-86820cd84f77", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65572025-04-30T06:13:19.763ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
65582025-04-30T06:13:19.764ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4fa847e-6aa1-4d37-a4f5-c52d3cf1ea8c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65592025-04-30T06:13:19.764ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
65602025-04-30T06:13:19.767ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eee8249-cdc0-4510-a25a-9d5dee66b6c1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65612025-04-30T06:13:19.767ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
65622025-04-30T06:13:19.769ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "828e8d5a-bd88-4b1f-a720-218a8b29da92", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65632025-04-30T06:13:19.769ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
65642025-04-30T06:13:19.772ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "471df76b-b995-4f46-a76e-7e76f52191c6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65652025-04-30T06:13:19.772ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
65662025-04-30T06:13:19.777ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c207caf5-5fb1-4beb-9992-bbd17c940d21", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65672025-04-30T06:13:19.777ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
65682025-04-30T06:13:19.783ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca67a08a-c9fb-4c35-adb5-a9bacbf3262d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65692025-04-30T06:13:19.783ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
65702025-04-30T06:13:19.788ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea07ce42-cbe5-4267-8a83-95f446f4a9b5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65712025-04-30T06:13:19.789ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
65722025-04-30T06:13:19.793ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d85f742d-a139-4f02-b571-7c1a6085410f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65732025-04-30T06:13:19.793ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
65742025-04-30T06:13:19.796ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "18021d84-f7ab-44e5-9b38-a740b513d539", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
65752025-04-30T06:13:19.797ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
65762025-04-30T06:13:19.799ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4910442-42ea-486a-b6b2-b56ae06582fa", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65772025-04-30T06:13:19.799ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
65782025-04-30T06:13:19.801ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8fb1d3be-e130-4f41-89af-864d7db5528f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:18 GMT"} })
65792025-04-30T06:13:19.801ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
65802025-04-30T06:13:19.801ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65812025-04-30T06:13:19.801ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
65822025-04-30T06:13:19.801ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
65832025-04-30T06:13:19.802ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
65842025-04-30T06:13:19.813ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65852025-04-30T06:13:19.819ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
65862025-04-30T06:13:19.819ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
65872025-04-30T06:13:19.819ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65882025-04-30T06:13:19.819ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
65892025-04-30T06:13:19.819ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
65902025-04-30T06:13:19.820ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
65912025-04-30T06:13:19.823ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
65922025-04-30T06:13:19.829ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
65932025-04-30T06:13:19.830ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
65942025-04-30T06:13:19.830ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
65952025-04-30T06:13:19.830ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
65962025-04-30T06:13:19.831ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
65972025-04-30T06:13:19.881ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
65982025-04-30T06:13:19.881ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
65992025-04-30T06:13:19.881ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66002025-04-30T06:13:19.881ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
66012025-04-30T06:13:19.882ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66022025-04-30T06:13:19.903ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
66032025-04-30T06:13:19.904ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
66042025-04-30T06:13:19.905ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:60744 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
66052025-04-30T06:13:19.908ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:60744 req_id = 4c0a88be-6f6f-4efe-ae6c-688751a822c0 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
66062025-04-30T06:13:19.909ZINFOomicron-dev (dropshot): request completed latency_us = 793 local_addr = [::1]:61881 method = GET remote_addr = [::1]:60744 req_id = 4c0a88be-6f6f-4efe-ae6c-688751a822c0 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
66072025-04-30T06:13:19.913ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c0a88be-6f6f-4efe-ae6c-688751a822c0", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
66082025-04-30T06:13:19.929ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66092025-04-30T06:13:19.960ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
66102025-04-30T06:13:19.961ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66112025-04-30T06:13:19.961ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
66122025-04-30T06:13:19.963ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66132025-04-30T06:13:19.987ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
66142025-04-30T06:13:19.987ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
66152025-04-30T06:13:19.987ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
66162025-04-30T06:13:19.995ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66172025-04-30T06:13:19.995ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
66182025-04-30T06:13:19.995ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
66192025-04-30T06:13:19.995ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66202025-04-30T06:13:19.996ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66212025-04-30T06:13:19.996ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66222025-04-30T06:13:19.997ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66232025-04-30T06:13:19.999ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66242025-04-30T06:13:20.003ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66252025-04-30T06:13:20.010ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
66262025-04-30T06:13:20.010ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66272025-04-30T06:13:20.011ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66282025-04-30T06:13:20.011ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66292025-04-30T06:13:20.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
66302025-04-30T06:13:20.090ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
66312025-04-30T06:13:20.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
66322025-04-30T06:13:20.092ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
66332025-04-30T06:13:20.097ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
66342025-04-30T06:13:20.097ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
66352025-04-30T06:13:20.104ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
66362025-04-30T06:13:20.104ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
66372025-04-30T06:13:20.125ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
66382025-04-30T06:13:20.125ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
66392025-04-30T06:13:20.125ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 566.797878ms iteration = 19 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
66402025-04-30T06:13:20.126ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 20 reason = Signaled
66412025-04-30T06:13:20.126ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
66422025-04-30T06:13:20.126ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66432025-04-30T06:13:20.126ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66442025-04-30T06:13:20.126ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66452025-04-30T06:13:20.128ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66462025-04-30T06:13:20.132ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66472025-04-30T06:13:20.139ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
66482025-04-30T06:13:20.139ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
66492025-04-30T06:13:20.140ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
66502025-04-30T06:13:20.140ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
66512025-04-30T06:13:20.140ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
66522025-04-30T06:13:20.145ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
66532025-04-30T06:13:20.145ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
66542025-04-30T06:13:20.147ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 88 a4 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55230 req_id = f7fd13c4-c7a3-4da1-8f64-e1389369ba70
66552025-04-30T06:13:20.185ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 34980,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55230 req_id = f7fd13c4-c7a3-4da1-8f64-e1389369ba70
66562025-04-30T06:13:20.185ZDEBGomicron-dev (store): query key key = _mgs._tcp
66572025-04-30T06:13:20.185ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
66582025-04-30T06:13:20.185ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55230 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f7fd13c4-c7a3-4da1-8f64-e1389369ba70
66592025-04-30T06:13:20.185ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2464, tv_nsec: 983150306 } })
66602025-04-30T06:13:20.185ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
66612025-04-30T06:13:20.185ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d1 1f 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60597 req_id = c3e6665b-294f-4157-affb-33a0eeea2af0
66622025-04-30T06:13:20.185ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53535,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60597 req_id = c3e6665b-294f-4157-affb-33a0eeea2af0
66632025-04-30T06:13:20.186ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
66642025-04-30T06:13:20.186ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60597 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = c3e6665b-294f-4157-affb-33a0eeea2af0
66652025-04-30T06:13:20.192ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
66662025-04-30T06:13:20.192ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
66672025-04-30T06:13:20.192ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66682025-04-30T06:13:20.192ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
66692025-04-30T06:13:20.192ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
66702025-04-30T06:13:20.194ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66712025-04-30T06:13:20.198ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
66722025-04-30T06:13:20.204ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
66732025-04-30T06:13:20.204ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66742025-04-30T06:13:20.204ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
66752025-04-30T06:13:20.205ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66762025-04-30T06:13:20.208ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
66772025-04-30T06:13:20.208ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
66782025-04-30T06:13:20.209ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
66792025-04-30T06:13:20.239ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
66802025-04-30T06:13:20.253ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
66812025-04-30T06:13:20.255ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 81 06 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63311 req_id = c62da88e-7b50-48f7-acdd-0aece3e451b9
66822025-04-30T06:13:20.255ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33030,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63311 req_id = c62da88e-7b50-48f7-acdd-0aece3e451b9
66832025-04-30T06:13:20.256ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
66842025-04-30T06:13:20.256ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 33030, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63311 req_id = c62da88e-7b50-48f7-acdd-0aece3e451b9
66852025-04-30T06:13:20.257ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
66862025-04-30T06:13:20.258ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
66872025-04-30T06:13:20.258ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
66882025-04-30T06:13:20.262ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb306267-4fff-4a10-8b27-2515e224cf70", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
66892025-04-30T06:13:20.264ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
66902025-04-30T06:13:20.265ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "747f9355-7cda-4f27-9692-cc92a48f75f0", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
66912025-04-30T06:13:20.266ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
66922025-04-30T06:13:20.268ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51944710-7954-485d-9cd8-f3466f133b56", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
66932025-04-30T06:13:20.268ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
66942025-04-30T06:13:20.270ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd7ebb15-637d-49e5-b654-bcb69a3cbc86", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
66952025-04-30T06:13:20.270ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
66962025-04-30T06:13:20.272ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "70e35875-00ff-4bea-818c-40a74750db8c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
66972025-04-30T06:13:20.272ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
66982025-04-30T06:13:20.274ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e26947ef-3ee1-4e9c-a0d9-6d0a65aa23c7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
66992025-04-30T06:13:20.274ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
67002025-04-30T06:13:20.276ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d9461b0-86b3-4e9a-a260-137d9d8e96e1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67012025-04-30T06:13:20.276ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
67022025-04-30T06:13:20.278ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd95ff45-70d5-492f-8117-b0a17c06c426", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67032025-04-30T06:13:20.278ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
67042025-04-30T06:13:20.279ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60220ae5-4fad-49a3-a322-c0d1d07cef4b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67052025-04-30T06:13:20.279ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
67062025-04-30T06:13:20.280ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "00b1b50d-d2fc-408c-bfbe-68d8149b7fd8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67072025-04-30T06:13:20.281ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
67082025-04-30T06:13:20.283ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9058cee8-80b9-4744-92dd-19c59b2f5dee", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67092025-04-30T06:13:20.283ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
67102025-04-30T06:13:20.285ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "12f7c197-e9be-4d52-a2e0-7920b2f01a39", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67112025-04-30T06:13:20.285ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
67122025-04-30T06:13:20.286ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb700e31-60d4-4b7e-b264-f226726464e2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67132025-04-30T06:13:20.286ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
67142025-04-30T06:13:20.288ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "996dde27-aa4c-484a-ae05-87344b55933f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67152025-04-30T06:13:20.288ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
67162025-04-30T06:13:20.290ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5940d09-0c04-49a9-a6d9-b4e73cc251ea", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67172025-04-30T06:13:20.290ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
67182025-04-30T06:13:20.292ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "963f1fb2-443f-460a-b142-ff5aebf1b45c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67192025-04-30T06:13:20.292ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
67202025-04-30T06:13:20.292ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
67212025-04-30T06:13:20.292ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
67222025-04-30T06:13:20.293ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
67232025-04-30T06:13:20.293ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
67242025-04-30T06:13:20.294ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "290e96a9-045a-4531-bb8b-bac0f3e702f2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67252025-04-30T06:13:20.294ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
67262025-04-30T06:13:20.295ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67272025-04-30T06:13:20.299ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67282025-04-30T06:13:20.305ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "971ba312-9628-4471-95c6-1ec001f9a68a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67292025-04-30T06:13:20.305ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
67302025-04-30T06:13:20.307ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "20e6d5ec-5e51-4610-bedb-0138ca7aa6e5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67312025-04-30T06:13:20.307ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
67322025-04-30T06:13:20.308ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1ec11a3-066e-4f43-8158-46bee1dc36fe", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67332025-04-30T06:13:20.309ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
67342025-04-30T06:13:20.310ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "183fae94-f02b-4038-a524-fbf62084bd8b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67352025-04-30T06:13:20.310ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
67362025-04-30T06:13:20.315ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
67372025-04-30T06:13:20.315ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
67382025-04-30T06:13:20.315ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
67392025-04-30T06:13:20.315ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67402025-04-30T06:13:20.317ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fc13784-b555-4237-be93-1a2ee0b1ff59", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67412025-04-30T06:13:20.317ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
67422025-04-30T06:13:20.319ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b424fea-0d1b-45b3-bdef-c439329903e0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67432025-04-30T06:13:20.319ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
67442025-04-30T06:13:20.321ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
67452025-04-30T06:13:20.321ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
67462025-04-30T06:13:20.321ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7498350a-53ee-4766-af61-96c62621f121", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67472025-04-30T06:13:20.321ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
67482025-04-30T06:13:20.322ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
67492025-04-30T06:13:20.332ZDEBGomicron-dev (BackgroundTasks): activating background_task = blueprint_loader iteration = 3 reason = Timeout
67502025-04-30T06:13:20.332ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
67512025-04-30T06:13:20.332ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
67522025-04-30T06:13:20.332ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
67532025-04-30T06:13:20.332ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67542025-04-30T06:13:20.334ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "49759856-1f70-4a86-9356-a65eb87d0d47", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67552025-04-30T06:13:20.334ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
67562025-04-30T06:13:20.337ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67572025-04-30T06:13:20.338ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "915f739a-6b6a-47dc-95f9-0fa6ab99644c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67582025-04-30T06:13:20.339ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
67592025-04-30T06:13:20.355ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67602025-04-30T06:13:20.356ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b46497b-1945-4c87-b8af-0452bfd6a71d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67612025-04-30T06:13:20.356ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
67622025-04-30T06:13:20.360ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4223aa7d-1423-4b71-8cff-13d8df4f627e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67632025-04-30T06:13:20.361ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
67642025-04-30T06:13:20.361ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
67652025-04-30T06:13:20.361ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
67662025-04-30T06:13:20.361ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67672025-04-30T06:13:20.362ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67682025-04-30T06:13:20.365ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af7b0c05-35d4-4a02-98a0-f521ba78b524", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67692025-04-30T06:13:20.368ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
67702025-04-30T06:13:20.368ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bfc4639-5ca6-4323-a678-d4a0347a6fdf, lookup_type: ById(1bfc4639-5ca6-4323-a678-d4a0347a6fdf) }
67712025-04-30T06:13:20.368ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
67722025-04-30T06:13:20.368ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
67732025-04-30T06:13:20.368ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67742025-04-30T06:13:20.369ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67752025-04-30T06:13:20.371ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
67762025-04-30T06:13:20.373ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
67772025-04-30T06:13:20.381ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bfc4639-5ca6-4323-a678-d4a0347a6fdf, lookup_type: ById(1bfc4639-5ca6-4323-a678-d4a0347a6fdf) } result = Ok(())
67782025-04-30T06:13:20.381ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
67792025-04-30T06:13:20.381ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
67802025-04-30T06:13:20.382ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
67812025-04-30T06:13:20.386ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5926d100-38a7-495c-878c-f089ab2b7294", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67822025-04-30T06:13:20.387ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
67832025-04-30T06:13:20.389ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2dcce3f-e563-43e9-9e65-74922157aaa9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67842025-04-30T06:13:20.389ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
67852025-04-30T06:13:20.391ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6de25b46-f199-409f-8e59-171a0cb7b3ab", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67862025-04-30T06:13:20.391ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
67872025-04-30T06:13:20.393ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba68d70d-f98d-4380-b695-8b8c52af8904", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67882025-04-30T06:13:20.393ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
67892025-04-30T06:13:20.396ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3636d467-0c1b-47e6-aaa4-b9abbc911e1a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67902025-04-30T06:13:20.396ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
67912025-04-30T06:13:20.398ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9668fc5b-c8ca-43a8-a8a9-cb3384e200bb", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67922025-04-30T06:13:20.398ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
67932025-04-30T06:13:20.401ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "23d2ad5f-69bf-40d8-aff7-8ddb25c280c2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
67942025-04-30T06:13:20.401ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
67952025-04-30T06:13:20.404ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "445a107d-f682-4088-8dfa-e9b1519c4635", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67962025-04-30T06:13:20.404ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
67972025-04-30T06:13:20.407ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e7d152e-1fc5-4d2d-a835-8e0c32a02369", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
67982025-04-30T06:13:20.407ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
67992025-04-30T06:13:20.411ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b59e4fb0-7b98-4fb1-8a2a-02e804cdcad2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
68002025-04-30T06:13:20.412ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
68012025-04-30T06:13:20.414ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "da3fd33c-0c4a-4f9c-b130-bfd448ebe3dc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
68022025-04-30T06:13:20.414ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
68032025-04-30T06:13:20.417ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "821a3cb6-9acb-4a91-8cb5-9ea0e6d9810b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
68042025-04-30T06:13:20.418ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
68052025-04-30T06:13:20.419ZDEBGomicron-dev (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bfc4639-5ca6-4323-a678-d4a0347a6fdf original_time_created = 2025-04-30 06:13:06.783824 UTC target_id = 1bfc4639-5ca6-4323-a678-d4a0347a6fdf time_created = 2025-04-30 06:13:06.783824 UTC
68062025-04-30T06:13:20.419ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 87.470229ms iteration = 3 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1bfc4639-5ca6-4323-a678-d4a0347a6fdf","time_created":"2025-04-30T06:13:06.783824Z"}
68072025-04-30T06:13:20.421ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "acaab106-7b3d-48ed-9733-b4e99bfba9ac", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
68082025-04-30T06:13:20.422ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
68092025-04-30T06:13:20.425ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6da03a4f-228e-42c0-b7ba-5e61fc99f98e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
68102025-04-30T06:13:20.425ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
68112025-04-30T06:13:20.425ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
68122025-04-30T06:13:20.427ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9dba0c4-787f-45f8-b896-943b9b54c479", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
68132025-04-30T06:13:20.427ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68142025-04-30T06:13:20.427ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68152025-04-30T06:13:20.427ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
68162025-04-30T06:13:20.427ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
68172025-04-30T06:13:20.429ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68182025-04-30T06:13:20.433ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "880caf92-7655-4c84-9e98-54b99b8beb8e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:19 GMT"} })
68192025-04-30T06:13:20.433ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68202025-04-30T06:13:20.443ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
68212025-04-30T06:13:20.443ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68222025-04-30T06:13:20.443ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
68232025-04-30T06:13:20.443ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68242025-04-30T06:13:20.444ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
68252025-04-30T06:13:20.444ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68262025-04-30T06:13:20.444ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68272025-04-30T06:13:20.444ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68282025-04-30T06:13:20.445ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68292025-04-30T06:13:20.451ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68302025-04-30T06:13:20.451ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
68312025-04-30T06:13:20.453ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68322025-04-30T06:13:20.453ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68332025-04-30T06:13:20.467ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
68342025-04-30T06:13:20.467ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
68352025-04-30T06:13:20.467ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68362025-04-30T06:13:20.467ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68372025-04-30T06:13:20.467ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68382025-04-30T06:13:20.467ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68392025-04-30T06:13:20.489ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68402025-04-30T06:13:20.494ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
68412025-04-30T06:13:20.494ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68422025-04-30T06:13:20.494ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68432025-04-30T06:13:20.494ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68442025-04-30T06:13:20.551ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
68452025-04-30T06:13:20.602ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
68462025-04-30T06:13:20.602ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68472025-04-30T06:13:20.602ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68482025-04-30T06:13:20.602ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
68492025-04-30T06:13:20.606ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68502025-04-30T06:13:20.679ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
68512025-04-30T06:13:20.685ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
68522025-04-30T06:13:20.685ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
68532025-04-30T06:13:20.687ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:43907 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
68542025-04-30T06:13:20.688ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:43907 req_id = ee939da7-ae76-4c25-ab07-c1fe08f263cc sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
68552025-04-30T06:13:20.688ZINFOomicron-dev (dropshot): request completed latency_us = 495 local_addr = [::1]:61881 method = GET remote_addr = [::1]:43907 req_id = ee939da7-ae76-4c25-ab07-c1fe08f263cc response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
68562025-04-30T06:13:20.689ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68572025-04-30T06:13:20.689ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee939da7-ae76-4c25-ab07-c1fe08f263cc", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
68582025-04-30T06:13:20.689ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
68592025-04-30T06:13:20.690ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6e917be3-bbfe-490b-9043-ae077d0384ce uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
68602025-04-30T06:13:20.691ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6e917be3-bbfe-490b-9043-ae077d0384ce resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
68612025-04-30T06:13:20.691ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
68622025-04-30T06:13:20.691ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
68632025-04-30T06:13:20.691ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
68642025-04-30T06:13:20.692ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6e917be3-bbfe-490b-9043-ae077d0384ce roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
68652025-04-30T06:13:20.693ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
68662025-04-30T06:13:20.693ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
68672025-04-30T06:13:20.693ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68682025-04-30T06:13:20.693ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68692025-04-30T06:13:20.694ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68702025-04-30T06:13:20.695ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6e917be3-bbfe-490b-9043-ae077d0384ce resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
68712025-04-30T06:13:20.695ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
68722025-04-30T06:13:20.695ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68732025-04-30T06:13:20.695ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
68742025-04-30T06:13:20.696ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68752025-04-30T06:13:20.701ZINFOomicron-dev (dropshot_internal): request completed latency_us = 11247 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 6e917be3-bbfe-490b-9043-ae077d0384ce response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
68762025-04-30T06:13:20.702ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
68772025-04-30T06:13:20.702ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e917be3-bbfe-490b-9043-ae077d0384ce", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
68782025-04-30T06:13:20.702ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
68792025-04-30T06:13:20.703ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
68802025-04-30T06:13:20.703ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
68812025-04-30T06:13:20.703ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0edaa1bc-ca2f-43c7-8cbb-0fb10b7a6211 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
68822025-04-30T06:13:20.703ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
68832025-04-30T06:13:20.704ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0edaa1bc-ca2f-43c7-8cbb-0fb10b7a6211 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
68842025-04-30T06:13:20.704ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0edaa1bc-ca2f-43c7-8cbb-0fb10b7a6211 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
68852025-04-30T06:13:20.705ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0edaa1bc-ca2f-43c7-8cbb-0fb10b7a6211 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
68862025-04-30T06:13:20.713ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
68872025-04-30T06:13:20.714ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
68882025-04-30T06:13:20.714ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
68892025-04-30T06:13:20.716ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
68902025-04-30T06:13:20.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68912025-04-30T06:13:20.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68922025-04-30T06:13:20.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68932025-04-30T06:13:20.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68942025-04-30T06:13:20.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68952025-04-30T06:13:20.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68962025-04-30T06:13:20.722ZINFOomicron-dev (dropshot_internal): request completed latency_us = 18433 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0edaa1bc-ca2f-43c7-8cbb-0fb10b7a6211 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
68972025-04-30T06:13:20.722ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "0edaa1bc-ca2f-43c7-8cbb-0fb10b7a6211", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
68982025-04-30T06:13:20.722ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
68992025-04-30T06:13:20.722ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
69002025-04-30T06:13:20.722ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
69012025-04-30T06:13:20.763ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
69022025-04-30T06:13:20.775ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
69032025-04-30T06:13:20.775ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
69042025-04-30T06:13:20.775ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 649.800365ms iteration = 20 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
69052025-04-30T06:13:20.775ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 21 reason = Signaled
69062025-04-30T06:13:20.776ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
69072025-04-30T06:13:20.776ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
69082025-04-30T06:13:20.776ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
69092025-04-30T06:13:20.776ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
69102025-04-30T06:13:20.779ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
69112025-04-30T06:13:20.782ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69122025-04-30T06:13:20.790ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
69132025-04-30T06:13:20.790ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
69142025-04-30T06:13:20.790ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
69152025-04-30T06:13:20.790ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
69162025-04-30T06:13:20.794ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
69172025-04-30T06:13:20.794ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
69182025-04-30T06:13:20.796ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: be 03 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61154 req_id = 938e1122-3f74-4213-b13d-26eeccb4f17c
69192025-04-30T06:13:20.796ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 48643,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61154 req_id = 938e1122-3f74-4213-b13d-26eeccb4f17c
69202025-04-30T06:13:20.796ZDEBGomicron-dev (store): query key key = _mgs._tcp
69212025-04-30T06:13:20.796ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
69222025-04-30T06:13:20.796ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61154 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 938e1122-3f74-4213-b13d-26eeccb4f17c
69232025-04-30T06:13:20.797ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2465, tv_nsec: 596063376 } })
69242025-04-30T06:13:20.797ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
69252025-04-30T06:13:20.798ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ec fb 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50953 req_id = a29acfb7-fc3d-4dfc-8cd7-329da3f5bbc6
69262025-04-30T06:13:20.798ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60667,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50953 req_id = a29acfb7-fc3d-4dfc-8cd7-329da3f5bbc6
69272025-04-30T06:13:20.799ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
69282025-04-30T06:13:20.799ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50953 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a29acfb7-fc3d-4dfc-8cd7-329da3f5bbc6
69292025-04-30T06:13:20.851ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
69302025-04-30T06:13:20.852ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
69312025-04-30T06:13:20.852ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
69322025-04-30T06:13:20.852ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69332025-04-30T06:13:20.852ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
69342025-04-30T06:13:20.853ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 08 84 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50675 req_id = 3ed51bb1-7a03-4671-a737-da23b316b148
69352025-04-30T06:13:20.853ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2180,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50675 req_id = 3ed51bb1-7a03-4671-a737-da23b316b148
69362025-04-30T06:13:20.853ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69372025-04-30T06:13:20.853ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
69382025-04-30T06:13:20.854ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 2180, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50675 req_id = 3ed51bb1-7a03-4671-a737-da23b316b148
69392025-04-30T06:13:20.855ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
69402025-04-30T06:13:20.855ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
69412025-04-30T06:13:20.855ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
69422025-04-30T06:13:20.858ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f505111-b32b-4d4d-b71d-403c5da348cd", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69432025-04-30T06:13:20.858ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
69442025-04-30T06:13:20.859ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
69452025-04-30T06:13:20.865ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c468d5c-2694-488e-bfe9-f6dfa28d46bc", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69462025-04-30T06:13:20.865ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
69472025-04-30T06:13:20.865ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69482025-04-30T06:13:20.865ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
69492025-04-30T06:13:20.865ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
69502025-04-30T06:13:20.866ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69512025-04-30T06:13:20.869ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
69522025-04-30T06:13:20.869ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
69532025-04-30T06:13:20.870ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
69542025-04-30T06:13:20.871ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "020030fe-44b6-4244-9c39-f180e0c12032", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69552025-04-30T06:13:20.871ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
69562025-04-30T06:13:20.877ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6268a3fd-abb4-4c9b-8fc3-cdf27bd78ec5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69572025-04-30T06:13:20.877ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
69582025-04-30T06:13:20.877ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1885cf78-c2f5-4e4d-95ab-9754131e08b4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69592025-04-30T06:13:20.877ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
69602025-04-30T06:13:20.878ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b5001f1-aa2a-4894-b1a3-c42555676645", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69612025-04-30T06:13:20.878ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
69622025-04-30T06:13:20.880ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aecc0007-3c9d-4b1d-857e-1ee4f3f36a73", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69632025-04-30T06:13:20.880ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
69642025-04-30T06:13:20.882ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "366782ff-8ad0-41df-845a-7e1461003869", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69652025-04-30T06:13:20.882ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
69662025-04-30T06:13:20.883ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a50f6f24-f690-485e-a7e7-f9e3fc910657", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69672025-04-30T06:13:20.883ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
69682025-04-30T06:13:20.884ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2597b2e-35e4-4f6c-a8b0-ad51ea6e724b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69692025-04-30T06:13:20.884ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
69702025-04-30T06:13:20.885ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7fe70bd-0384-456a-b03b-37d92d4f3eea", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69712025-04-30T06:13:20.885ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
69722025-04-30T06:13:20.886ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ed4fa2c-c603-495a-9196-d949c85b7996", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69732025-04-30T06:13:20.886ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
69742025-04-30T06:13:20.887ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "79f090ef-b4d5-4f69-92a7-4431163079b2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69752025-04-30T06:13:20.887ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
69762025-04-30T06:13:20.889ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "981c4c36-e44d-4294-86a1-eb9d6f9eee1a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69772025-04-30T06:13:20.889ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
69782025-04-30T06:13:20.891ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "837db2ce-f450-4fb6-bbcf-ab51cfb1aab5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69792025-04-30T06:13:20.891ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
69802025-04-30T06:13:20.894ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35b534cd-a112-46a8-887b-7b8ebd40bba3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69812025-04-30T06:13:20.894ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
69822025-04-30T06:13:20.903ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6382a34f-9526-4e90-b5ff-74d5972a55ab", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69832025-04-30T06:13:20.903ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
69842025-04-30T06:13:20.910ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "279f290b-7062-47db-a2e3-65c177e659ba", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69852025-04-30T06:13:20.911ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
69862025-04-30T06:13:20.919ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
69872025-04-30T06:13:20.921ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "86fc3741-5749-4d80-b1c1-50bb878aa7e1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69882025-04-30T06:13:20.921ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
69892025-04-30T06:13:20.924ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "80b4f311-a62a-46a5-aa4b-4fa2ec36bd11", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69902025-04-30T06:13:20.924ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
69912025-04-30T06:13:20.927ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bf51659-c929-46b8-9fd2-d0e0ea4eb1fa", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69922025-04-30T06:13:20.927ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
69932025-04-30T06:13:20.929ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b02d6e6-8a37-4954-977b-990ae8dd4da5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69942025-04-30T06:13:20.929ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
69952025-04-30T06:13:20.935ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe3f2e6c-ec68-4641-a55c-356ce5726d96", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69962025-04-30T06:13:20.935ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
69972025-04-30T06:13:20.937ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25795a14-5340-4fe4-a4da-5b8566775b5b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
69982025-04-30T06:13:20.937ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
69992025-04-30T06:13:20.940ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3027af88-f96d-40e0-bea8-f6df4ab36295", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70002025-04-30T06:13:20.940ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
70012025-04-30T06:13:20.943ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "430b02bc-30c5-416e-8de5-512fe7eaf44a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70022025-04-30T06:13:20.944ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
70032025-04-30T06:13:20.945ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1368f9d-aa6f-4654-a2f0-88fb6413762b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70042025-04-30T06:13:20.946ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
70052025-04-30T06:13:20.947ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a516902-defd-4826-89f1-147e7458ea92", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70062025-04-30T06:13:20.947ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
70072025-04-30T06:13:20.948ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0242faa-d936-43cd-a56b-66352f3af813", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70082025-04-30T06:13:20.949ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
70092025-04-30T06:13:20.950ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80ab5e72-5dc0-4e37-95dd-f7a7ce8c572f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70102025-04-30T06:13:20.950ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
70112025-04-30T06:13:20.951ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "785d4637-4dc2-4404-b9a1-c226446d6d10", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70122025-04-30T06:13:20.951ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
70132025-04-30T06:13:20.953ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "32cdddd0-47f8-4f19-a450-a0afa6dbafb1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70142025-04-30T06:13:20.953ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
70152025-04-30T06:13:20.954ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4ecf9bd-cb14-4f08-9200-83d648c32e97", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70162025-04-30T06:13:20.955ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
70172025-04-30T06:13:20.957ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "66fe59ac-bbfa-46f9-b137-4d03b7058cd5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70182025-04-30T06:13:20.957ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
70192025-04-30T06:13:20.958ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e528f972-00d1-4ed4-9ef8-3680e35a21e6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70202025-04-30T06:13:20.959ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
70212025-04-30T06:13:20.960ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ffab10b-37eb-4b4f-a925-21c0147faad9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70222025-04-30T06:13:20.962ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
70232025-04-30T06:13:20.965ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ab1c050-a678-4635-bf88-af73f0dc8e9e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70242025-04-30T06:13:20.965ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
70252025-04-30T06:13:20.971ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
70262025-04-30T06:13:20.971ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70272025-04-30T06:13:20.971ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70282025-04-30T06:13:20.971ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
70292025-04-30T06:13:20.972ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70302025-04-30T06:13:20.975ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccbf7e4c-a3d0-495d-84fe-0c8607ea400c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70312025-04-30T06:13:20.975ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70322025-04-30T06:13:20.978ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
70332025-04-30T06:13:20.982ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "693dadf0-9c7f-4f8f-abd8-464a48f331f6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70342025-04-30T06:13:20.982ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
70352025-04-30T06:13:20.983ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
70362025-04-30T06:13:20.983ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70372025-04-30T06:13:20.983ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
70382025-04-30T06:13:20.983ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70392025-04-30T06:13:20.985ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba81313e-1a3f-454f-be1c-909c0463ac03", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70402025-04-30T06:13:20.992ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
70412025-04-30T06:13:20.992ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70422025-04-30T06:13:20.992ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
70432025-04-30T06:13:20.992ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70442025-04-30T06:13:20.992ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a99c7d5-5535-4936-acb6-e67d7bd9a588", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70452025-04-30T06:13:20.992ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
70462025-04-30T06:13:20.992ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f4d233c-db96-4391-b79b-9f327bdcbd2b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70472025-04-30T06:13:20.992ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
70482025-04-30T06:13:20.992ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5da7d9c4-0d05-4a94-a6f3-4fe6e6012d6f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70492025-04-30T06:13:20.992ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
70502025-04-30T06:13:20.993ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2fe84fd-e1f7-4cee-9a85-5893e2b018ef", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70512025-04-30T06:13:20.993ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
70522025-04-30T06:13:20.996ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "59d2edd0-c8b7-4c4a-b485-66ceca03c5b1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
70532025-04-30T06:13:20.997ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
70542025-04-30T06:13:20.997ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70552025-04-30T06:13:20.997ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70562025-04-30T06:13:20.997ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70572025-04-30T06:13:20.999ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70582025-04-30T06:13:21.006ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70592025-04-30T06:13:21.013ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
70602025-04-30T06:13:21.014ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
70612025-04-30T06:13:21.014ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70622025-04-30T06:13:21.014ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70632025-04-30T06:13:21.014ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70642025-04-30T06:13:21.015ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70652025-04-30T06:13:21.018ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70662025-04-30T06:13:21.022ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
70672025-04-30T06:13:21.026ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
70682025-04-30T06:13:21.026ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
70692025-04-30T06:13:21.026ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
70702025-04-30T06:13:21.027ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
70712025-04-30T06:13:21.085ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
70722025-04-30T06:13:21.085ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
70732025-04-30T06:13:21.085ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70742025-04-30T06:13:21.085ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
70752025-04-30T06:13:21.086ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70762025-04-30T06:13:21.094ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
70772025-04-30T06:13:21.095ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
70782025-04-30T06:13:21.095ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
70792025-04-30T06:13:21.104ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
70802025-04-30T06:13:21.104ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
70812025-04-30T06:13:21.105ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
70822025-04-30T06:13:21.105ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
70832025-04-30T06:13:21.105ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
70842025-04-30T06:13:21.106ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
70852025-04-30T06:13:21.112ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
70862025-04-30T06:13:21.113ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:64720 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
70872025-04-30T06:13:21.116ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
70882025-04-30T06:13:21.123ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:64720 req_id = be8241b9-27db-4e8a-b55f-c96569562110 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
70892025-04-30T06:13:21.123ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
70902025-04-30T06:13:21.123ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70912025-04-30T06:13:21.123ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
70922025-04-30T06:13:21.123ZINFOomicron-dev (dropshot): request completed latency_us = 535 local_addr = [::1]:61881 method = GET remote_addr = [::1]:64720 req_id = be8241b9-27db-4e8a-b55f-c96569562110 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
70932025-04-30T06:13:21.124ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "be8241b9-27db-4e8a-b55f-c96569562110", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
70942025-04-30T06:13:21.124ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
70952025-04-30T06:13:21.127ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
70962025-04-30T06:13:21.127ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
70972025-04-30T06:13:21.127ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
70982025-04-30T06:13:21.127ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
70992025-04-30T06:13:21.127ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
71002025-04-30T06:13:21.127ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71012025-04-30T06:13:21.129ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
71022025-04-30T06:13:21.129ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
71032025-04-30T06:13:21.129ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
71042025-04-30T06:13:21.129ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71052025-04-30T06:13:21.130ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
71062025-04-30T06:13:21.140ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71072025-04-30T06:13:21.149ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
71082025-04-30T06:13:21.167ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
71092025-04-30T06:13:21.168ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
71102025-04-30T06:13:21.168ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71112025-04-30T06:13:21.169ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
71122025-04-30T06:13:21.196ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
71132025-04-30T06:13:21.212ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
71142025-04-30T06:13:21.213ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
71152025-04-30T06:13:21.213ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 437.49986ms iteration = 21 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
71162025-04-30T06:13:21.213ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 22 reason = Signaled
71172025-04-30T06:13:21.213ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
71182025-04-30T06:13:21.213ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
71192025-04-30T06:13:21.213ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
71202025-04-30T06:13:21.213ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71212025-04-30T06:13:21.214ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
71222025-04-30T06:13:21.218ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71232025-04-30T06:13:21.218ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
71242025-04-30T06:13:21.218ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
71252025-04-30T06:13:21.218ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71262025-04-30T06:13:21.218ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
71272025-04-30T06:13:21.219ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71282025-04-30T06:13:21.222ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
71292025-04-30T06:13:21.222ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
71302025-04-30T06:13:21.222ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
71312025-04-30T06:13:21.222ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
71322025-04-30T06:13:21.227ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
71332025-04-30T06:13:21.229ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
71342025-04-30T06:13:21.229ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
71352025-04-30T06:13:21.229ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: da 0e 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:49930 req_id = 94685124-4e03-48e6-a0cb-29e35186c5ef
71362025-04-30T06:13:21.229ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55822,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49930 req_id = 94685124-4e03-48e6-a0cb-29e35186c5ef
71372025-04-30T06:13:21.230ZDEBGomicron-dev (store): query key key = _mgs._tcp
71382025-04-30T06:13:21.230ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
71392025-04-30T06:13:21.230ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:49930 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 94685124-4e03-48e6-a0cb-29e35186c5ef
71402025-04-30T06:13:21.231ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2466, tv_nsec: 30194293 } })
71412025-04-30T06:13:21.231ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
71422025-04-30T06:13:21.231ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d9 5d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .]..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52109 req_id = 7ef6bcc5-2bc9-43e2-9764-8828a3b61b26
71432025-04-30T06:13:21.232ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55645,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52109 req_id = 7ef6bcc5-2bc9-43e2-9764-8828a3b61b26
71442025-04-30T06:13:21.232ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
71452025-04-30T06:13:21.232ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52109 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7ef6bcc5-2bc9-43e2-9764-8828a3b61b26
71462025-04-30T06:13:21.236ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
71472025-04-30T06:13:21.236ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71482025-04-30T06:13:21.236ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
71492025-04-30T06:13:21.236ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71502025-04-30T06:13:21.240ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
71512025-04-30T06:13:21.240ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
71522025-04-30T06:13:21.241ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
71532025-04-30T06:13:21.278ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
71542025-04-30T06:13:21.279ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 9f b2 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58980 req_id = 19d78c4a-66ff-4bb9-b58a-2ede18dc9c49
71552025-04-30T06:13:21.279ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40882,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58980 req_id = 19d78c4a-66ff-4bb9-b58a-2ede18dc9c49
71562025-04-30T06:13:21.280ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
71572025-04-30T06:13:21.280ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 40882, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58980 req_id = 19d78c4a-66ff-4bb9-b58a-2ede18dc9c49
71582025-04-30T06:13:21.280ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
71592025-04-30T06:13:21.280ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
71602025-04-30T06:13:21.280ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
71612025-04-30T06:13:21.284ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c01a4a3-5471-43dd-a0ae-bc32076e3d23", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
71622025-04-30T06:13:21.285ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
71632025-04-30T06:13:21.287ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc3bf3cf-05a3-4b01-a5df-62423f66b89c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
71642025-04-30T06:13:21.287ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
71652025-04-30T06:13:21.290ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b694718-eb7a-476d-8423-3dda6aff0610", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
71662025-04-30T06:13:21.290ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
71672025-04-30T06:13:21.293ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23943581-a127-472a-895e-07e560e45c13", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
71682025-04-30T06:13:21.293ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
71692025-04-30T06:13:21.297ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ed5d4a1-07ff-441b-8fd2-d2bef94c5d3c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
71702025-04-30T06:13:21.298ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
71712025-04-30T06:13:21.301ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d5f79e2-3876-4403-88f3-b3147bfd0b2c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
71722025-04-30T06:13:21.302ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
71732025-04-30T06:13:21.304ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f230f9d-e9cd-4759-b7ce-fdc4f19fdb1d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
71742025-04-30T06:13:21.304ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
71752025-04-30T06:13:21.304ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
71762025-04-30T06:13:21.318ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6a43a85-1b52-4572-8e3f-26ba38ea35b8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
71772025-04-30T06:13:21.319ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
71782025-04-30T06:13:21.322ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60f40cc1-237e-4bfd-8f6b-ba9b99f18e31", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
71792025-04-30T06:13:21.324ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
71802025-04-30T06:13:21.325ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3042eba-ffe1-4c83-8898-6cf381176625", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
71812025-04-30T06:13:21.326ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
71822025-04-30T06:13:21.328ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2946c3f2-d4d3-428b-bfc6-32a133d7983e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
71832025-04-30T06:13:21.329ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
71842025-04-30T06:13:21.330ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1930313-28f8-448a-b5f8-643fe8c50e81", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
71852025-04-30T06:13:21.330ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
71862025-04-30T06:13:21.332ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5bfc691-a8a1-499c-8f35-0ff5e3619874", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
71872025-04-30T06:13:21.333ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
71882025-04-30T06:13:21.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ada4ad7e-e31e-4df1-a733-9f6017caf156", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
71892025-04-30T06:13:21.336ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
71902025-04-30T06:13:21.340ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "86cc9466-0c09-40bc-a65f-a0dced15ce6d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
71912025-04-30T06:13:21.341ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
71922025-04-30T06:13:21.346ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3845ac83-e98e-4076-b661-67dbb0ccb784", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
71932025-04-30T06:13:21.346ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
71942025-04-30T06:13:21.350ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b59ccaca-5607-4a79-a092-92fc90d0f350", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
71952025-04-30T06:13:21.350ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
71962025-04-30T06:13:21.355ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f7afe78-dfb8-4432-9e6a-1c3e33815d29", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
71972025-04-30T06:13:21.355ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
71982025-04-30T06:13:21.356ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
71992025-04-30T06:13:21.356ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72002025-04-30T06:13:21.356ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72012025-04-30T06:13:21.356ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
72022025-04-30T06:13:21.359ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72032025-04-30T06:13:21.359ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0733fd4-4d8b-46e8-8097-eafca197d88c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72042025-04-30T06:13:21.359ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
72052025-04-30T06:13:21.363ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44c71ee6-cf80-44a9-8616-9ce129cb262c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
72062025-04-30T06:13:21.363ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
72072025-04-30T06:13:21.365ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e934b2ec-8d92-4046-b352-1ddd042ec8b2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
72082025-04-30T06:13:21.365ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
72092025-04-30T06:13:21.366ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72102025-04-30T06:13:21.367ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0edfdb2f-09db-4914-bc48-ca816e997f34", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
72112025-04-30T06:13:21.368ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
72122025-04-30T06:13:21.370ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "478995ba-78aa-41cf-b659-81a49a2442ec", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
72132025-04-30T06:13:21.370ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
72142025-04-30T06:13:21.376ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "93f73c23-f6b5-4116-9188-d96d86f23333", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72152025-04-30T06:13:21.376ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
72162025-04-30T06:13:21.376ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72172025-04-30T06:13:21.376ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
72182025-04-30T06:13:21.376ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
72192025-04-30T06:13:21.377ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72202025-04-30T06:13:21.381ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d97e73f-9af0-4b4a-807f-f13a98ac066c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72212025-04-30T06:13:21.382ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
72222025-04-30T06:13:21.385ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a482adb8-e0f6-431f-9c20-e528e2cfee87", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72232025-04-30T06:13:21.385ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
72242025-04-30T06:13:21.387ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c511fcb7-3007-43e9-ade7-df071c6bd5e0", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72252025-04-30T06:13:21.387ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
72262025-04-30T06:13:21.389ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72272025-04-30T06:13:21.389ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
72282025-04-30T06:13:21.389ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e796ca06-ffe9-4bee-9b3c-8b969fbd9a47", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72292025-04-30T06:13:21.389ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
72302025-04-30T06:13:21.389ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72312025-04-30T06:13:21.400ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4ba35fb-72ee-4296-9589-280c89f674c8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72322025-04-30T06:13:21.400ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
72332025-04-30T06:13:21.406ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b164751-e03c-4a22-b01f-1e935c8e9062", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72342025-04-30T06:13:21.406ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
72352025-04-30T06:13:21.410ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6494552c-b6fa-49e1-95c1-ac07d186ebed", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72362025-04-30T06:13:21.411ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
72372025-04-30T06:13:21.413ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "da3f9637-86e1-41a2-85c4-d19b3544f609", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
72382025-04-30T06:13:21.413ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
72392025-04-30T06:13:21.423ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a89749a-504a-41f4-804d-4357ea22aacf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
72402025-04-30T06:13:21.424ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
72412025-04-30T06:13:21.427ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c5148cd-c481-4aae-8792-5084e5f86540", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72422025-04-30T06:13:21.427ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
72432025-04-30T06:13:21.430ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0f46c46-a5a8-407e-80bd-84e687ae999c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72442025-04-30T06:13:21.430ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
72452025-04-30T06:13:21.456ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
72462025-04-30T06:13:21.465ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c515346c-f1ff-4264-a8c3-0f97f77c652e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72472025-04-30T06:13:21.465ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
72482025-04-30T06:13:21.491ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a458f76b-f880-41b9-83db-6ae95f228298", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72492025-04-30T06:13:21.491ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
72502025-04-30T06:13:21.493ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36c7a9e7-1981-49fa-be02-1c67e94669a1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72512025-04-30T06:13:21.493ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
72522025-04-30T06:13:21.496ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7be640c2-4791-4f8e-a52c-610731f8e5af", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72532025-04-30T06:13:21.496ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
72542025-04-30T06:13:21.499ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4748745-7c1d-4e56-bf66-42f32fd4ff8a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72552025-04-30T06:13:21.500ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
72562025-04-30T06:13:21.521ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d97208ab-5ec1-459e-a32d-6fc7cd98dd84", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72572025-04-30T06:13:21.521ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
72582025-04-30T06:13:21.521ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
72592025-04-30T06:13:21.521ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72602025-04-30T06:13:21.521ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72612025-04-30T06:13:21.521ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
72622025-04-30T06:13:21.523ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72632025-04-30T06:13:21.525ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ffd9b7c-6205-4c12-9be6-c6a01d8aa55e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72642025-04-30T06:13:21.525ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
72652025-04-30T06:13:21.527ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3e9055c-d3ed-49cb-842d-8b3ddde10fc6", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
72662025-04-30T06:13:21.529ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
72672025-04-30T06:13:21.531ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99e31d53-085d-46c2-8e4e-d606710c88f5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72682025-04-30T06:13:21.531ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
72692025-04-30T06:13:21.532ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72702025-04-30T06:13:21.534ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bae8c09-7bd8-454d-8aea-5d2fc47f1f84", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
72712025-04-30T06:13:21.535ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
72722025-04-30T06:13:21.535ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72732025-04-30T06:13:21.535ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72742025-04-30T06:13:21.535ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72752025-04-30T06:13:21.537ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72762025-04-30T06:13:21.540ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72772025-04-30T06:13:21.547ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
72782025-04-30T06:13:21.547ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72792025-04-30T06:13:21.547ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
72802025-04-30T06:13:21.548ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72812025-04-30T06:13:21.550ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
72822025-04-30T06:13:21.550ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
72832025-04-30T06:13:21.551ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
72842025-04-30T06:13:21.551ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72852025-04-30T06:13:21.551ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72862025-04-30T06:13:21.553ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72872025-04-30T06:13:21.558ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
72882025-04-30T06:13:21.572ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
72892025-04-30T06:13:21.572ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
72902025-04-30T06:13:21.575ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
72912025-04-30T06:13:21.578ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
72922025-04-30T06:13:21.578ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
72932025-04-30T06:13:21.578ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
72942025-04-30T06:13:21.579ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
72952025-04-30T06:13:21.641ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
72962025-04-30T06:13:21.673ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
72972025-04-30T06:13:21.673ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
72982025-04-30T06:13:21.674ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:37613 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
72992025-04-30T06:13:21.675ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:37613 req_id = c0a5d72f-af3c-4286-8f72-f8174adf1e47 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
73002025-04-30T06:13:21.675ZINFOomicron-dev (dropshot): request completed latency_us = 734 local_addr = [::1]:61881 method = GET remote_addr = [::1]:37613 req_id = c0a5d72f-af3c-4286-8f72-f8174adf1e47 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
73012025-04-30T06:13:21.676ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0a5d72f-af3c-4286-8f72-f8174adf1e47", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:20 GMT"} })
73022025-04-30T06:13:21.678ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
73032025-04-30T06:13:21.678ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
73042025-04-30T06:13:21.679ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
73052025-04-30T06:13:21.681ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
73062025-04-30T06:13:21.681ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
73072025-04-30T06:13:21.681ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
73082025-04-30T06:13:21.681ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73092025-04-30T06:13:21.682ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73102025-04-30T06:13:21.690ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73112025-04-30T06:13:21.692ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
73122025-04-30T06:13:21.692ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
73132025-04-30T06:13:21.692ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
73142025-04-30T06:13:21.693ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
73152025-04-30T06:13:21.694ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
73162025-04-30T06:13:21.699ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
73172025-04-30T06:13:21.699ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
73182025-04-30T06:13:21.699ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73192025-04-30T06:13:21.700ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73202025-04-30T06:13:21.703ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73212025-04-30T06:13:21.714ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
73222025-04-30T06:13:21.714ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
73232025-04-30T06:13:21.714ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
73242025-04-30T06:13:21.716ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
73252025-04-30T06:13:21.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
73262025-04-30T06:13:21.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
73272025-04-30T06:13:21.721ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
73282025-04-30T06:13:21.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
73292025-04-30T06:13:21.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
73302025-04-30T06:13:21.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
73312025-04-30T06:13:21.733ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
73322025-04-30T06:13:21.734ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
73332025-04-30T06:13:21.734ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
73342025-04-30T06:13:21.734ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 520.720221ms iteration = 22 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
73352025-04-30T06:13:21.734ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 23 reason = Signaled
73362025-04-30T06:13:21.734ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
73372025-04-30T06:13:21.734ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
73382025-04-30T06:13:21.734ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
73392025-04-30T06:13:21.734ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
73402025-04-30T06:13:21.734ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73412025-04-30T06:13:21.735ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73422025-04-30T06:13:21.737ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
73432025-04-30T06:13:21.739ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
73442025-04-30T06:13:21.743ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
73452025-04-30T06:13:21.743ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
73462025-04-30T06:13:21.743ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
73472025-04-30T06:13:21.743ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
73482025-04-30T06:13:21.747ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
73492025-04-30T06:13:21.747ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
73502025-04-30T06:13:21.747ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a4 49 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .I..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53309 req_id = e3492c84-e1a2-41c6-83b4-35c16898cea7
73512025-04-30T06:13:21.748ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42057,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53309 req_id = e3492c84-e1a2-41c6-83b4-35c16898cea7
73522025-04-30T06:13:21.749ZDEBGomicron-dev (store): query key key = _mgs._tcp
73532025-04-30T06:13:21.749ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
73542025-04-30T06:13:21.749ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53309 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e3492c84-e1a2-41c6-83b4-35c16898cea7
73552025-04-30T06:13:21.750ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2466, tv_nsec: 549193065 } })
73562025-04-30T06:13:21.750ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
73572025-04-30T06:13:21.750ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ab 91 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51810 req_id = a93c89cc-53f8-4b16-a51d-cc17287d781b
73582025-04-30T06:13:21.751ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 43921,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51810 req_id = a93c89cc-53f8-4b16-a51d-cc17287d781b
73592025-04-30T06:13:21.751ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
73602025-04-30T06:13:21.751ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51810 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a93c89cc-53f8-4b16-a51d-cc17287d781b
73612025-04-30T06:13:21.785ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
73622025-04-30T06:13:21.788ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
73632025-04-30T06:13:21.789ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 11 4b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .K..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61295 req_id = 1e66dc48-a501-4156-adf9-fa240775d7a9
73642025-04-30T06:13:21.789ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4427,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61295 req_id = 1e66dc48-a501-4156-adf9-fa240775d7a9
73652025-04-30T06:13:21.789ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
73662025-04-30T06:13:21.789ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 4427, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61295 req_id = 1e66dc48-a501-4156-adf9-fa240775d7a9
73672025-04-30T06:13:21.789ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
73682025-04-30T06:13:21.789ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
73692025-04-30T06:13:21.789ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
73702025-04-30T06:13:21.792ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cbbde5f-65e0-45d0-9e12-0e813b0e38c0", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73712025-04-30T06:13:21.792ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
73722025-04-30T06:13:21.794ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "00dba0f4-ad08-4358-8885-900679e196d9", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73732025-04-30T06:13:21.794ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
73742025-04-30T06:13:21.796ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8daf6ee-d3b0-46a1-8a5e-412f2993c474", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73752025-04-30T06:13:21.796ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
73762025-04-30T06:13:21.798ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "60e2be89-5678-4f18-8985-778f5d7e14b3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73772025-04-30T06:13:21.799ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
73782025-04-30T06:13:21.800ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "de70d87c-10b0-4850-ab26-acbb6dbb5881", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73792025-04-30T06:13:21.801ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
73802025-04-30T06:13:21.803ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8f39f03-b15e-4989-af0e-11337eaa6e06", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73812025-04-30T06:13:21.803ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
73822025-04-30T06:13:21.805ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "54cd4ad1-57f4-40dd-a371-4b35878343c0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73832025-04-30T06:13:21.805ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
73842025-04-30T06:13:21.807ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dec7a1b0-9d3c-41ee-b5ab-fab42b22df51", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73852025-04-30T06:13:21.807ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
73862025-04-30T06:13:21.808ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e847605c-56f2-45a8-ab66-05816615e20b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73872025-04-30T06:13:21.808ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
73882025-04-30T06:13:21.809ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a54c4356-b086-4a1b-a5a9-a92e757b2bba", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73892025-04-30T06:13:21.809ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
73902025-04-30T06:13:21.810ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db6a9efb-3aba-40a8-b12e-662c373c10ff", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73912025-04-30T06:13:21.811ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
73922025-04-30T06:13:21.812ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfbd94d4-c061-4550-a7a5-adf656876ea8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73932025-04-30T06:13:21.813ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
73942025-04-30T06:13:21.814ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8a1328e-c835-4add-a0e3-2e7829ad04dd", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73952025-04-30T06:13:21.814ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
73962025-04-30T06:13:21.816ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bce077aa-79f0-4211-8a3a-8858f8936b13", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73972025-04-30T06:13:21.816ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
73982025-04-30T06:13:21.818ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fa1fa23-6dd1-4353-a8db-23910af444dc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
73992025-04-30T06:13:21.818ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
74002025-04-30T06:13:21.820ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd3d625a-de57-4868-b068-16554283e1ac", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74012025-04-30T06:13:21.820ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
74022025-04-30T06:13:21.823ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e8a98ec-52d2-4504-bb19-4df201d79d98", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74032025-04-30T06:13:21.823ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
74042025-04-30T06:13:21.825ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "04a29c56-8118-4b77-a762-695380a38e2d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74052025-04-30T06:13:21.825ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
74062025-04-30T06:13:21.827ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "168f75bc-5fbd-40eb-9132-34613d918484", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74072025-04-30T06:13:21.827ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
74082025-04-30T06:13:21.828ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef3f992b-6f9b-4a3a-af66-80accdd2f37b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74092025-04-30T06:13:21.829ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
74102025-04-30T06:13:21.831ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44feacda-ebe9-4535-b91e-6a23081eebdf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74112025-04-30T06:13:21.831ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
74122025-04-30T06:13:21.833ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "547f9ae7-2f57-4ccf-9c1f-8d6884da4cd8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74132025-04-30T06:13:21.834ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
74142025-04-30T06:13:21.835ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "87010daf-ae28-443f-8712-244e14f0a834", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74152025-04-30T06:13:21.835ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
74162025-04-30T06:13:21.837ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "530db906-fdd0-43a6-bac6-13ec73a492ce", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74172025-04-30T06:13:21.837ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
74182025-04-30T06:13:21.839ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6230ba45-dede-4d81-964e-900a235b4452", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74192025-04-30T06:13:21.839ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
74202025-04-30T06:13:21.839ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
74212025-04-30T06:13:21.839ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
74222025-04-30T06:13:21.840ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74232025-04-30T06:13:21.840ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
74242025-04-30T06:13:21.841ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "382feae1-451d-43de-938e-5beb0d1812fb", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74252025-04-30T06:13:21.841ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
74262025-04-30T06:13:21.841ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74272025-04-30T06:13:21.848ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a15a5bb-14d9-4c88-8034-223df0ae4bef", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74282025-04-30T06:13:21.849ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
74292025-04-30T06:13:21.851ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72996d5e-b338-47c8-bd85-3776586605af", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74302025-04-30T06:13:21.851ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
74312025-04-30T06:13:21.853ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a006f82-7a09-4361-bd34-fc2f2398456d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74322025-04-30T06:13:21.853ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
74332025-04-30T06:13:21.856ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6510443c-5bec-435a-9ae1-63d19406c494", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74342025-04-30T06:13:21.856ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
74352025-04-30T06:13:21.857ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "153482a8-3e73-4503-85d7-65e731a92c90", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74362025-04-30T06:13:21.857ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
74372025-04-30T06:13:21.858ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1580710-0b9a-4096-9401-8e112e838ca3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74382025-04-30T06:13:21.858ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
74392025-04-30T06:13:21.859ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74402025-04-30T06:13:21.859ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e97b1dd0-ae44-496f-9b17-c0c91104ddf5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74412025-04-30T06:13:21.860ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
74422025-04-30T06:13:21.861ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4eed15d-eb21-4184-a980-93be65599666", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74432025-04-30T06:13:21.861ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
74442025-04-30T06:13:21.863ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c8311e3-a72e-4dfe-9800-bc6814a80d87", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74452025-04-30T06:13:21.863ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
74462025-04-30T06:13:21.866ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a180851f-9e7a-4ecf-8aa2-77f750178f39", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74472025-04-30T06:13:21.866ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
74482025-04-30T06:13:21.866ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74492025-04-30T06:13:21.866ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
74502025-04-30T06:13:21.866ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
74512025-04-30T06:13:21.866ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74522025-04-30T06:13:21.869ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74532025-04-30T06:13:21.869ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
74542025-04-30T06:13:21.870ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74552025-04-30T06:13:21.871ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c992d51d-5e83-4fa0-8e5d-3b07c380e901", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74562025-04-30T06:13:21.871ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
74572025-04-30T06:13:21.878ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7b0729d-4e04-4593-982c-947b733d486f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74582025-04-30T06:13:21.878ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
74592025-04-30T06:13:21.880ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c7764ae-2072-4bcc-9da7-0ef72c424055", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74602025-04-30T06:13:21.880ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
74612025-04-30T06:13:21.882ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2255a258-e9dc-4a86-aa5d-9c0415713d67", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74622025-04-30T06:13:21.882ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
74632025-04-30T06:13:21.884ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46f15a07-4751-42bb-8e8a-acdc55e960aa", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74642025-04-30T06:13:21.884ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
74652025-04-30T06:13:21.884ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "390c8c7f-76ca-41d5-ade2-bce9ea5bdbda", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74662025-04-30T06:13:21.885ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
74672025-04-30T06:13:21.887ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "23d2e0ef-ddc3-4482-9f4a-50472bc5e5bf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74682025-04-30T06:13:21.887ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
74692025-04-30T06:13:21.894ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "faac2c4b-a24e-4a54-9e57-772d49b615eb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74702025-04-30T06:13:21.894ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
74712025-04-30T06:13:21.896ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad5c20f1-1bb6-4eff-a59d-ed201f375a17", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
74722025-04-30T06:13:21.896ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
74732025-04-30T06:13:21.896ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
74742025-04-30T06:13:21.896ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
74752025-04-30T06:13:21.896ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
74762025-04-30T06:13:21.897ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
74772025-04-30T06:13:21.902ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74782025-04-30T06:13:21.912ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
74792025-04-30T06:13:21.912ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
74802025-04-30T06:13:21.913ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
74812025-04-30T06:13:21.913ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
74822025-04-30T06:13:21.913ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
74832025-04-30T06:13:21.914ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
74842025-04-30T06:13:21.918ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74852025-04-30T06:13:21.920ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
74862025-04-30T06:13:21.926ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
74872025-04-30T06:13:21.926ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
74882025-04-30T06:13:21.926ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
74892025-04-30T06:13:21.928ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
74902025-04-30T06:13:21.972ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
74912025-04-30T06:13:21.972ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
74922025-04-30T06:13:21.972ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
74932025-04-30T06:13:21.972ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
74942025-04-30T06:13:21.973ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
74952025-04-30T06:13:21.995ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
74962025-04-30T06:13:21.995ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
74972025-04-30T06:13:21.998ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:62341 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
74982025-04-30T06:13:21.999ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
74992025-04-30T06:13:21.999ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:62341 req_id = ba28ead5-68d2-4e8e-bb80-0b6341287acf sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
75002025-04-30T06:13:21.999ZINFOomicron-dev (dropshot): request completed latency_us = 473 local_addr = [::1]:61881 method = GET remote_addr = [::1]:62341 req_id = ba28ead5-68d2-4e8e-bb80-0b6341287acf response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
75012025-04-30T06:13:22.000ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba28ead5-68d2-4e8e-bb80-0b6341287acf", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
75022025-04-30T06:13:22.005ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
75032025-04-30T06:13:22.005ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
75042025-04-30T06:13:22.005ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
75052025-04-30T06:13:22.009ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
75062025-04-30T06:13:22.009ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75072025-04-30T06:13:22.009ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75082025-04-30T06:13:22.009ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75092025-04-30T06:13:22.009ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
75102025-04-30T06:13:22.009ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75112025-04-30T06:13:22.010ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
75122025-04-30T06:13:22.010ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75132025-04-30T06:13:22.010ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75142025-04-30T06:13:22.013ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75152025-04-30T06:13:22.013ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75162025-04-30T06:13:22.013ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
75172025-04-30T06:13:22.014ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75182025-04-30T06:13:22.017ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
75192025-04-30T06:13:22.017ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75202025-04-30T06:13:22.017ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75212025-04-30T06:13:22.017ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75222025-04-30T06:13:22.045ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
75232025-04-30T06:13:22.045ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
75242025-04-30T06:13:22.046ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 311.260776ms iteration = 23 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
75252025-04-30T06:13:22.046ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 24 reason = Signaled
75262025-04-30T06:13:22.046ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
75272025-04-30T06:13:22.046ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75282025-04-30T06:13:22.046ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75292025-04-30T06:13:22.046ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75302025-04-30T06:13:22.047ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75312025-04-30T06:13:22.049ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75322025-04-30T06:13:22.053ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
75332025-04-30T06:13:22.053ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
75342025-04-30T06:13:22.053ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
75352025-04-30T06:13:22.053ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
75362025-04-30T06:13:22.057ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
75372025-04-30T06:13:22.058ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
75382025-04-30T06:13:22.058ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: fa cb 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:50938 req_id = ee1f0205-33b3-4733-8d50-750b61d01abe
75392025-04-30T06:13:22.058ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64203,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50938 req_id = ee1f0205-33b3-4733-8d50-750b61d01abe
75402025-04-30T06:13:22.058ZDEBGomicron-dev (store): query key key = _mgs._tcp
75412025-04-30T06:13:22.058ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
75422025-04-30T06:13:22.059ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50938 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = ee1f0205-33b3-4733-8d50-750b61d01abe
75432025-04-30T06:13:22.059ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2466, tv_nsec: 858269498 } })
75442025-04-30T06:13:22.059ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
75452025-04-30T06:13:22.060ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ad 0b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53543 req_id = bdd7e6f5-82d2-469c-aa68-e4f5aa99505c
75462025-04-30T06:13:22.060ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44299,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53543 req_id = bdd7e6f5-82d2-469c-aa68-e4f5aa99505c
75472025-04-30T06:13:22.060ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
75482025-04-30T06:13:22.060ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53543 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = bdd7e6f5-82d2-469c-aa68-e4f5aa99505c
75492025-04-30T06:13:22.127ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
75502025-04-30T06:13:22.127ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
75512025-04-30T06:13:22.127ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
75522025-04-30T06:13:22.127ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
75532025-04-30T06:13:22.127ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
75542025-04-30T06:13:22.130ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
75552025-04-30T06:13:22.130ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
75562025-04-30T06:13:22.133ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
75572025-04-30T06:13:22.133ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
75582025-04-30T06:13:22.141ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
75592025-04-30T06:13:22.142ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 6c d9 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c l............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62062 req_id = 450d2217-85bc-4230-a2a3-f5ddc375a077
75602025-04-30T06:13:22.142ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27865,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62062 req_id = 450d2217-85bc-4230-a2a3-f5ddc375a077
75612025-04-30T06:13:22.142ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
75622025-04-30T06:13:22.142ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 27865, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62062 req_id = 450d2217-85bc-4230-a2a3-f5ddc375a077
75632025-04-30T06:13:22.142ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
75642025-04-30T06:13:22.142ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
75652025-04-30T06:13:22.142ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
75662025-04-30T06:13:22.145ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "7683bbc1-0138-420f-b35b-6b9d15b40a98", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
75672025-04-30T06:13:22.145ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
75682025-04-30T06:13:22.147ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7883e290-be79-4db0-99e1-f40d07d67833", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
75692025-04-30T06:13:22.147ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
75702025-04-30T06:13:22.149ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbf795e7-3b3b-4dc3-b3d3-eef669dabc5a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
75712025-04-30T06:13:22.150ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
75722025-04-30T06:13:22.151ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee5dd932-bfd5-4425-8dbd-357bcdfff113", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
75732025-04-30T06:13:22.152ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
75742025-04-30T06:13:22.157ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bbae8d7-f9ac-45af-847f-50b685d44e3e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
75752025-04-30T06:13:22.157ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
75762025-04-30T06:13:22.157ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "848259a6-6964-4a4b-8b2c-16aec66f716b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
75772025-04-30T06:13:22.157ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
75782025-04-30T06:13:22.159ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d88d413-2845-48c8-884d-c2dd4d59000e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
75792025-04-30T06:13:22.160ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
75802025-04-30T06:13:22.164ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dae31307-a2d0-4aad-b72d-1f20defa76c7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
75812025-04-30T06:13:22.164ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
75822025-04-30T06:13:22.165ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fd45603-93d1-417c-84bb-2e90d83ed145", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
75832025-04-30T06:13:22.172ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
75842025-04-30T06:13:22.174ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
75852025-04-30T06:13:22.174ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
75862025-04-30T06:13:22.174ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75872025-04-30T06:13:22.174ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
75882025-04-30T06:13:22.175ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
75892025-04-30T06:13:22.176ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fd11f72-e1c1-41d5-a565-0fcf64f82a68", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
75902025-04-30T06:13:22.176ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
75912025-04-30T06:13:22.178ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3687b5f8-dce5-4b4e-afe9-2019af553aad", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
75922025-04-30T06:13:22.178ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
75932025-04-30T06:13:22.179ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d064db14-f495-48f4-bb14-1b20d9fa7369", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
75942025-04-30T06:13:22.179ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
75952025-04-30T06:13:22.179ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
75962025-04-30T06:13:22.181ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "61c839bd-6001-4bb2-bff2-336d3e11dc97", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
75972025-04-30T06:13:22.184ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
75982025-04-30T06:13:22.184ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
75992025-04-30T06:13:22.184ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
76002025-04-30T06:13:22.184ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76012025-04-30T06:13:22.187ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
76022025-04-30T06:13:22.190ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76032025-04-30T06:13:22.190ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
76042025-04-30T06:13:22.190ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76052025-04-30T06:13:22.192ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0dada2aa-2c96-4f36-88b9-3e5284e27a06", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76062025-04-30T06:13:22.192ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
76072025-04-30T06:13:22.195ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2b6d89b-f9f8-422f-8b82-469bdf215fca", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76082025-04-30T06:13:22.195ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
76092025-04-30T06:13:22.198ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2122ec0-9186-4cc8-b33b-2fa8604f8fdb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76102025-04-30T06:13:22.199ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
76112025-04-30T06:13:22.205ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b6fdf2f-196c-41e0-a4a1-04ef422d1303", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76122025-04-30T06:13:22.206ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
76132025-04-30T06:13:22.209ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e04851ac-7f63-4ec3-98b5-98b5ef10fe45", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76142025-04-30T06:13:22.209ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
76152025-04-30T06:13:22.213ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d75350c-b090-4109-a75c-1afbf56d3171", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76162025-04-30T06:13:22.213ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
76172025-04-30T06:13:22.214ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8654c891-dbc2-4d7c-9ec4-7a1bcb40ce6a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76182025-04-30T06:13:22.214ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
76192025-04-30T06:13:22.216ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af2a39d9-37e3-4b14-8e7b-5ba78efcbb04", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
76202025-04-30T06:13:22.216ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
76212025-04-30T06:13:22.223ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2ea4c6d-131b-4404-8a23-8558862d7223", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76222025-04-30T06:13:22.223ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
76232025-04-30T06:13:22.231ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
76242025-04-30T06:13:22.232ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f56b0508-67ea-43bb-8de6-023654ed45d3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76252025-04-30T06:13:22.234ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
76262025-04-30T06:13:22.260ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
76272025-04-30T06:13:22.262ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2674356-c6b1-4449-98a4-841f22ee6d2d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
76282025-04-30T06:13:22.263ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
76292025-04-30T06:13:22.266ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbf5d95a-f17b-489e-b721-1934fa5f8da9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
76302025-04-30T06:13:22.267ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
76312025-04-30T06:13:22.275ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9816d783-0ef6-4d1d-ab53-59821e719e10", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
76322025-04-30T06:13:22.277ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
76332025-04-30T06:13:22.283ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f45d100-77bf-4856-adf6-0d244a07cca2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
76342025-04-30T06:13:22.284ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
76352025-04-30T06:13:22.290ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1248378b-ead9-45ba-a277-fea83c5da08c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
76362025-04-30T06:13:22.291ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
76372025-04-30T06:13:22.307ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a06eccda-42ad-4409-996c-5d3b0e711573", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
76382025-04-30T06:13:22.338ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
76392025-04-30T06:13:22.339ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
76402025-04-30T06:13:22.339ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
76412025-04-30T06:13:22.339ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76422025-04-30T06:13:22.339ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
76432025-04-30T06:13:22.340ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76442025-04-30T06:13:22.342ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6739a89e-5650-4426-bac7-1d404dd2af28", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76452025-04-30T06:13:22.343ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
76462025-04-30T06:13:22.343ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0752e02-49f7-4b9d-aa6a-83fa0acd67bd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76472025-04-30T06:13:22.344ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
76482025-04-30T06:13:22.345ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0217d0ad-499b-475b-93e7-6dbaca549d68", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76492025-04-30T06:13:22.345ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
76502025-04-30T06:13:22.346ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "267d6aec-4f48-4cd4-aabc-8ad90d68c83c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76512025-04-30T06:13:22.346ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
76522025-04-30T06:13:22.346ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76532025-04-30T06:13:22.348ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "38c33b23-1913-4658-889b-886b136bfff9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76542025-04-30T06:13:22.348ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
76552025-04-30T06:13:22.349ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "115fd9a9-6cb9-4149-a88a-a226ae2be2d2", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76562025-04-30T06:13:22.349ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
76572025-04-30T06:13:22.351ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7f7c492-60f1-4150-8df5-c120f2e7e2dd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76582025-04-30T06:13:22.351ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
76592025-04-30T06:13:22.352ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
76602025-04-30T06:13:22.352ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76612025-04-30T06:13:22.352ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
76622025-04-30T06:13:22.353ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76632025-04-30T06:13:22.353ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7214edb-11d6-42bc-97a4-ebbdd2691878", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76642025-04-30T06:13:22.353ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
76652025-04-30T06:13:22.356ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
76662025-04-30T06:13:22.356ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
76672025-04-30T06:13:22.356ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5df5e015-c979-4561-aa06-8d66c9ecd498", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76682025-04-30T06:13:22.356ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
76692025-04-30T06:13:22.358ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
76702025-04-30T06:13:22.359ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68b207aa-adad-4f46-bc30-e3bc6a1c4424", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76712025-04-30T06:13:22.359ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
76722025-04-30T06:13:22.363ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec89f37b-3a79-4bc7-9da7-6465844a1799", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76732025-04-30T06:13:22.363ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
76742025-04-30T06:13:22.372ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0525b85-034e-4a25-8e0a-fb87f07a4ad7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76752025-04-30T06:13:22.372ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
76762025-04-30T06:13:22.377ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "44e376da-aea5-486e-a507-41e933a1a1aa", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76772025-04-30T06:13:22.377ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
76782025-04-30T06:13:22.381ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3a7870a-d854-41fc-92c5-85ec619659af", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76792025-04-30T06:13:22.382ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
76802025-04-30T06:13:22.384ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99d7f011-0a48-4e2b-99e1-2e173d884fce", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76812025-04-30T06:13:22.385ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
76822025-04-30T06:13:22.399ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4e2a570-a7f0-406b-9f58-f000fc894ad5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
76832025-04-30T06:13:22.399ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
76842025-04-30T06:13:22.399ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
76852025-04-30T06:13:22.399ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
76862025-04-30T06:13:22.399ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
76872025-04-30T06:13:22.400ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
76882025-04-30T06:13:22.410ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76892025-04-30T06:13:22.414ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
76902025-04-30T06:13:22.430ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
76912025-04-30T06:13:22.431ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
76922025-04-30T06:13:22.431ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
76932025-04-30T06:13:22.431ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
76942025-04-30T06:13:22.431ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
76952025-04-30T06:13:22.437ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
76962025-04-30T06:13:22.457ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
76972025-04-30T06:13:22.462ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
76982025-04-30T06:13:22.463ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
76992025-04-30T06:13:22.463ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77002025-04-30T06:13:22.464ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77012025-04-30T06:13:22.466ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
77022025-04-30T06:13:22.466ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77032025-04-30T06:13:22.467ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77042025-04-30T06:13:22.467ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
77052025-04-30T06:13:22.469ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77062025-04-30T06:13:22.517ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
77072025-04-30T06:13:22.517ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
77082025-04-30T06:13:22.518ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:45519 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
77092025-04-30T06:13:22.519ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:45519 req_id = e5e3bb57-1897-496e-821d-4f7b89aa67e7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
77102025-04-30T06:13:22.520ZINFOomicron-dev (dropshot): request completed latency_us = 825 local_addr = [::1]:61881 method = GET remote_addr = [::1]:45519 req_id = e5e3bb57-1897-496e-821d-4f7b89aa67e7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
77112025-04-30T06:13:22.520ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5e3bb57-1897-496e-821d-4f7b89aa67e7", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
77122025-04-30T06:13:22.523ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77132025-04-30T06:13:22.523ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
77142025-04-30T06:13:22.523ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
77152025-04-30T06:13:22.523ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
77162025-04-30T06:13:22.526ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
77172025-04-30T06:13:22.526ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77182025-04-30T06:13:22.526ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77192025-04-30T06:13:22.526ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77202025-04-30T06:13:22.527ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77212025-04-30T06:13:22.530ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
77222025-04-30T06:13:22.530ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77232025-04-30T06:13:22.530ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
77242025-04-30T06:13:22.531ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77252025-04-30T06:13:22.531ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77262025-04-30T06:13:22.538ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
77272025-04-30T06:13:22.538ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77282025-04-30T06:13:22.538ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77292025-04-30T06:13:22.539ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77302025-04-30T06:13:22.543ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77312025-04-30T06:13:22.543ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
77322025-04-30T06:13:22.546ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77332025-04-30T06:13:22.602ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
77342025-04-30T06:13:22.602ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
77352025-04-30T06:13:22.602ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 556.179461ms iteration = 24 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
77362025-04-30T06:13:22.602ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 25 reason = Signaled
77372025-04-30T06:13:22.603ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
77382025-04-30T06:13:22.603ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77392025-04-30T06:13:22.603ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77402025-04-30T06:13:22.603ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77412025-04-30T06:13:22.603ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
77422025-04-30T06:13:22.604ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77432025-04-30T06:13:22.607ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77442025-04-30T06:13:22.612ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
77452025-04-30T06:13:22.612ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
77462025-04-30T06:13:22.612ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
77472025-04-30T06:13:22.612ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
77482025-04-30T06:13:22.617ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
77492025-04-30T06:13:22.617ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
77502025-04-30T06:13:22.618ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 63 58 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 cX..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58259 req_id = 3617a459-5baa-4482-8b20-874011dc6d43
77512025-04-30T06:13:22.619ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25432,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58259 req_id = 3617a459-5baa-4482-8b20-874011dc6d43
77522025-04-30T06:13:22.619ZDEBGomicron-dev (store): query key key = _mgs._tcp
77532025-04-30T06:13:22.619ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
77542025-04-30T06:13:22.619ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58259 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 3617a459-5baa-4482-8b20-874011dc6d43
77552025-04-30T06:13:22.619ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2467, tv_nsec: 418873786 } })
77562025-04-30T06:13:22.620ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
77572025-04-30T06:13:22.620ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 56 ae 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e V...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57898 req_id = 41fa7464-57be-49f1-8318-82c8cae86786
77582025-04-30T06:13:22.621ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22190,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57898 req_id = 41fa7464-57be-49f1-8318-82c8cae86786
77592025-04-30T06:13:22.621ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
77602025-04-30T06:13:22.621ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57898 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 41fa7464-57be-49f1-8318-82c8cae86786
77612025-04-30T06:13:22.656ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
77622025-04-30T06:13:22.656ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
77632025-04-30T06:13:22.656ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77642025-04-30T06:13:22.656ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
77652025-04-30T06:13:22.657ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77662025-04-30T06:13:22.663ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
77672025-04-30T06:13:22.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
77682025-04-30T06:13:22.680ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
77692025-04-30T06:13:22.681ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 04056a3b-21f7-4c3f-aaf5-76cc40f74f23 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
77702025-04-30T06:13:22.681ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 04056a3b-21f7-4c3f-aaf5-76cc40f74f23 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
77712025-04-30T06:13:22.681ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 04056a3b-21f7-4c3f-aaf5-76cc40f74f23 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
77722025-04-30T06:13:22.682ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 04056a3b-21f7-4c3f-aaf5-76cc40f74f23 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
77732025-04-30T06:13:22.684ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
77742025-04-30T06:13:22.684ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77752025-04-30T06:13:22.684ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
77762025-04-30T06:13:22.684ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77772025-04-30T06:13:22.687ZINFOomicron-dev (dropshot_internal): request completed latency_us = 5834 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 04056a3b-21f7-4c3f-aaf5-76cc40f74f23 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
77782025-04-30T06:13:22.687ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "04056a3b-21f7-4c3f-aaf5-76cc40f74f23", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
77792025-04-30T06:13:22.687ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
77802025-04-30T06:13:22.689ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 31c8ce3b-cee2-4741-a4fe-30da1e5e1486 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
77812025-04-30T06:13:22.689ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 31c8ce3b-cee2-4741-a4fe-30da1e5e1486 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
77822025-04-30T06:13:22.689ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 31c8ce3b-cee2-4741-a4fe-30da1e5e1486 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
77832025-04-30T06:13:22.689ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
77842025-04-30T06:13:22.689ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
77852025-04-30T06:13:22.689ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
77862025-04-30T06:13:22.689ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 31c8ce3b-cee2-4741-a4fe-30da1e5e1486 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
77872025-04-30T06:13:22.690ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 15 ba 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64984 req_id = 9a47e6c2-fdcf-44da-a05a-230de35c5afe
77882025-04-30T06:13:22.690ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5562,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64984 req_id = 9a47e6c2-fdcf-44da-a05a-230de35c5afe
77892025-04-30T06:13:22.690ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
77902025-04-30T06:13:22.690ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5562, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64984 req_id = 9a47e6c2-fdcf-44da-a05a-230de35c5afe
77912025-04-30T06:13:22.690ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
77922025-04-30T06:13:22.690ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
77932025-04-30T06:13:22.690ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
77942025-04-30T06:13:22.690ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
77952025-04-30T06:13:22.693ZINFOomicron-dev (dropshot_internal): request completed latency_us = 5480 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 31c8ce3b-cee2-4741-a4fe-30da1e5e1486 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
77962025-04-30T06:13:22.693ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "31c8ce3b-cee2-4741-a4fe-30da1e5e1486", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
77972025-04-30T06:13:22.694ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
77982025-04-30T06:13:22.694ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
77992025-04-30T06:13:22.694ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
78002025-04-30T06:13:22.697ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "76617c38-8cca-4f05-9d8c-27e1807835ce", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
78012025-04-30T06:13:22.697ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
78022025-04-30T06:13:22.700ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "838834d8-f91c-4ae6-9c7f-2275aea0c54d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78032025-04-30T06:13:22.701ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
78042025-04-30T06:13:22.704ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "52dbfbc0-1c47-4d43-a2a1-3db363634a79", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78052025-04-30T06:13:22.704ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
78062025-04-30T06:13:22.706ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c5e72cf-ec14-48b9-9751-9f980cd0683f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78072025-04-30T06:13:22.706ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
78082025-04-30T06:13:22.713ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "da976e79-b22c-4400-82ea-56c729f92af7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78092025-04-30T06:13:22.714ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
78102025-04-30T06:13:22.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
78112025-04-30T06:13:22.724ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c31e26bd-ef86-4ba0-864f-54842826c74a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
78122025-04-30T06:13:22.724ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
78132025-04-30T06:13:22.727ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "091d5269-7156-4b58-9c51-9e8c2599d3fb", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
78142025-04-30T06:13:22.727ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
78152025-04-30T06:13:22.729ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7021e4f-db63-4600-843d-2804b3d1e192", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
78162025-04-30T06:13:22.730ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
78172025-04-30T06:13:22.730ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b4a4865-bb88-426c-a4fa-ac9fce4c96f2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:21 GMT"} })
78182025-04-30T06:13:22.731ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
78192025-04-30T06:13:22.733ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e931c509-062c-4e23-8385-b00c30b64c5c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78202025-04-30T06:13:22.734ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
78212025-04-30T06:13:22.734ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd4ab4ad-75f7-4f0b-bed9-ea2572b83312", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78222025-04-30T06:13:22.734ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
78232025-04-30T06:13:22.734ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37329ccf-4817-4906-b2d4-bd6236c9fc9a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78242025-04-30T06:13:22.735ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
78252025-04-30T06:13:22.737ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98d1c1a8-7687-4bfd-8114-48f7cbefa779", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78262025-04-30T06:13:22.737ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
78272025-04-30T06:13:22.741ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85c54bf5-7c18-4470-b40a-a71ce6669cfb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78282025-04-30T06:13:22.741ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
78292025-04-30T06:13:22.742ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
78302025-04-30T06:13:22.744ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff68ec11-a1ae-4ae9-a96c-c21cfc91717b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78312025-04-30T06:13:22.744ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
78322025-04-30T06:13:22.747ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb0fbf35-4d40-48ee-a2d3-713564a985c2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78332025-04-30T06:13:22.747ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
78342025-04-30T06:13:22.751ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
78352025-04-30T06:13:22.753ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
78362025-04-30T06:13:22.755ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
78372025-04-30T06:13:22.755ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
78382025-04-30T06:13:22.755ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
78392025-04-30T06:13:22.756ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d158135a-0742-47b6-8cf3-e09dc4243df0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78402025-04-30T06:13:22.756ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
78412025-04-30T06:13:22.764ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a607dc0-52c4-4193-a359-05a160502115", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78422025-04-30T06:13:22.764ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
78432025-04-30T06:13:22.777ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3c6e428-fd2f-418c-be7b-b506280cb2a7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78442025-04-30T06:13:22.777ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
78452025-04-30T06:13:22.780ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "757408fa-56f6-41c3-aca2-a3b85127436c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78462025-04-30T06:13:22.780ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
78472025-04-30T06:13:22.782ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "85cc92ba-eb31-48ae-a677-fd9741538a89", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78482025-04-30T06:13:22.782ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
78492025-04-30T06:13:22.785ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7762c63e-0772-4349-b3e2-03a8dbde0921", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78502025-04-30T06:13:22.785ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
78512025-04-30T06:13:22.790ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "61fa4bab-c38b-4aea-bccb-0243adca79cb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78522025-04-30T06:13:22.790ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
78532025-04-30T06:13:22.795ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6502384f-8cc7-4d4f-913e-d2959a1d17c6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78542025-04-30T06:13:22.795ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
78552025-04-30T06:13:22.795ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
78562025-04-30T06:13:22.795ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
78572025-04-30T06:13:22.795ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78582025-04-30T06:13:22.796ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
78592025-04-30T06:13:22.797ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78602025-04-30T06:13:22.801ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8aca87f-ed67-4a90-bf9c-d24fa56be45a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78612025-04-30T06:13:22.801ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
78622025-04-30T06:13:22.802ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
78632025-04-30T06:13:22.808ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
78642025-04-30T06:13:22.808ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78652025-04-30T06:13:22.808ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
78662025-04-30T06:13:22.809ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78672025-04-30T06:13:22.813ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
78682025-04-30T06:13:22.813ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
78692025-04-30T06:13:22.814ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
78702025-04-30T06:13:22.814ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d77768f9-d9ec-4696-a928-fbe12c7f324d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78712025-04-30T06:13:22.815ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
78722025-04-30T06:13:22.829ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a59ef7d0-e0d6-4635-a04d-072bbdc47e3b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78732025-04-30T06:13:22.829ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
78742025-04-30T06:13:22.838ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c401d3c-e65b-463c-a2b8-33b6fef2bddf", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78752025-04-30T06:13:22.839ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
78762025-04-30T06:13:22.848ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d97b797-82c3-43b1-a27a-3432c550404b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78772025-04-30T06:13:22.849ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
78782025-04-30T06:13:22.853ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b13638bd-5ef9-4264-ab8c-0a18645080c4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78792025-04-30T06:13:22.853ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
78802025-04-30T06:13:22.854ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "269f5b9c-e19a-4afa-899b-7d3d1978830d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78812025-04-30T06:13:22.855ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
78822025-04-30T06:13:22.864ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "13e18de7-57fc-4210-9f91-37424caaf772", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78832025-04-30T06:13:22.864ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
78842025-04-30T06:13:22.866ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c849c677-9c76-4e0a-9186-0a4d2b16bda2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78852025-04-30T06:13:22.871ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
78862025-04-30T06:13:22.872ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
78872025-04-30T06:13:22.875ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "198d2b53-b038-48ae-9841-1d0576441f1f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78882025-04-30T06:13:22.875ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
78892025-04-30T06:13:22.880ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dae87975-0896-4366-8e4f-ca552e7812cd", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78902025-04-30T06:13:22.881ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
78912025-04-30T06:13:22.902ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1b35334-ef92-4b36-b050-3a9d10df3da9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78922025-04-30T06:13:22.902ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
78932025-04-30T06:13:22.905ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9c873c0-bbec-4325-9db4-a37ab21fc18d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78942025-04-30T06:13:22.905ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
78952025-04-30T06:13:22.907ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fc4428f-cd3e-4c25-ad08-8a076e372729", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78962025-04-30T06:13:22.907ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
78972025-04-30T06:13:22.912ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "72725c01-4392-40c4-9d74-0ae03682b62f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
78982025-04-30T06:13:22.912ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
78992025-04-30T06:13:22.914ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b2ae140-dbab-4351-a67e-a1818e2a0cd2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
79002025-04-30T06:13:22.915ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
79012025-04-30T06:13:22.919ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1055477-078f-4b2b-a974-29fc5973cc1e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
79022025-04-30T06:13:22.919ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
79032025-04-30T06:13:22.922ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eec3fa23-69a9-4aaf-a2f6-322a19ee7b9c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
79042025-04-30T06:13:22.922ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
79052025-04-30T06:13:22.923ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
79062025-04-30T06:13:22.923ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79072025-04-30T06:13:22.923ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79082025-04-30T06:13:22.923ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
79092025-04-30T06:13:22.924ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79102025-04-30T06:13:22.927ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8f976e7-92c6-4a8e-bf24-b83de33e5f7f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
79112025-04-30T06:13:22.928ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
79122025-04-30T06:13:22.929ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79132025-04-30T06:13:22.934ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca2e60d6-2e74-4922-a064-dae8eb2db805", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
79142025-04-30T06:13:22.934ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
79152025-04-30T06:13:22.937ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
79162025-04-30T06:13:22.937ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79172025-04-30T06:13:22.937ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
79182025-04-30T06:13:22.937ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "343ce2db-c0ea-49c8-b057-32a23b72ba2f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
79192025-04-30T06:13:22.937ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
79202025-04-30T06:13:22.937ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79212025-04-30T06:13:22.937ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79222025-04-30T06:13:22.937ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79232025-04-30T06:13:22.937ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79242025-04-30T06:13:22.938ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79252025-04-30T06:13:22.943ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79262025-04-30T06:13:22.943ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
79272025-04-30T06:13:22.944ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79282025-04-30T06:13:22.944ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79292025-04-30T06:13:22.954ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
79302025-04-30T06:13:22.954ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
79312025-04-30T06:13:22.954ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79322025-04-30T06:13:22.954ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79332025-04-30T06:13:22.954ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79342025-04-30T06:13:22.955ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79352025-04-30T06:13:22.961ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79362025-04-30T06:13:22.972ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
79372025-04-30T06:13:22.972ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79382025-04-30T06:13:22.972ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79392025-04-30T06:13:22.973ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79402025-04-30T06:13:22.991ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
79412025-04-30T06:13:23.042ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
79422025-04-30T06:13:23.042ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79432025-04-30T06:13:23.042ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79442025-04-30T06:13:23.042ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
79452025-04-30T06:13:23.044ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79462025-04-30T06:13:23.083ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
79472025-04-30T06:13:23.083ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
79482025-04-30T06:13:23.085ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:55250 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
79492025-04-30T06:13:23.086ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:55250 req_id = 26317e03-280d-4fd3-8f26-f0e6a449e495 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
79502025-04-30T06:13:23.087ZINFOomicron-dev (dropshot): request completed latency_us = 846 local_addr = [::1]:61881 method = GET remote_addr = [::1]:55250 req_id = 26317e03-280d-4fd3-8f26-f0e6a449e495 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
79512025-04-30T06:13:23.087ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "26317e03-280d-4fd3-8f26-f0e6a449e495", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
79522025-04-30T06:13:23.089ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79532025-04-30T06:13:23.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
79542025-04-30T06:13:23.089ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
79552025-04-30T06:13:23.091ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
79562025-04-30T06:13:23.092ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
79572025-04-30T06:13:23.093ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
79582025-04-30T06:13:23.093ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
79592025-04-30T06:13:23.093ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
79602025-04-30T06:13:23.096ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
79612025-04-30T06:13:23.096ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79622025-04-30T06:13:23.096ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
79632025-04-30T06:13:23.096ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
79642025-04-30T06:13:23.096ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
79652025-04-30T06:13:23.096ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79662025-04-30T06:13:23.099ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
79672025-04-30T06:13:23.099ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79682025-04-30T06:13:23.099ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
79692025-04-30T06:13:23.099ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
79702025-04-30T06:13:23.099ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79712025-04-30T06:13:23.099ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79722025-04-30T06:13:23.100ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
79732025-04-30T06:13:23.100ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79742025-04-30T06:13:23.102ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
79752025-04-30T06:13:23.102ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
79762025-04-30T06:13:23.116ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79772025-04-30T06:13:23.126ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
79782025-04-30T06:13:23.127ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79792025-04-30T06:13:23.127ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79802025-04-30T06:13:23.131ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79812025-04-30T06:13:23.159ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
79822025-04-30T06:13:23.184ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
79832025-04-30T06:13:23.184ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
79842025-04-30T06:13:23.184ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 581.586122ms iteration = 25 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
79852025-04-30T06:13:23.184ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 26 reason = Signaled
79862025-04-30T06:13:23.184ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
79872025-04-30T06:13:23.184ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
79882025-04-30T06:13:23.184ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79892025-04-30T06:13:23.184ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79902025-04-30T06:13:23.185ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79912025-04-30T06:13:23.188ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
79922025-04-30T06:13:23.193ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
79932025-04-30T06:13:23.195ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
79942025-04-30T06:13:23.195ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
79952025-04-30T06:13:23.195ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
79962025-04-30T06:13:23.196ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
79972025-04-30T06:13:23.200ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
79982025-04-30T06:13:23.200ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
79992025-04-30T06:13:23.202ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 72 9c 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 r............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62480 req_id = 95844e7f-5237-4682-916c-61efb26073cb
80002025-04-30T06:13:23.202ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29340,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62480 req_id = 95844e7f-5237-4682-916c-61efb26073cb
80012025-04-30T06:13:23.203ZDEBGomicron-dev (store): query key key = _mgs._tcp
80022025-04-30T06:13:23.203ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
80032025-04-30T06:13:23.203ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62480 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 95844e7f-5237-4682-916c-61efb26073cb
80042025-04-30T06:13:23.204ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2468, tv_nsec: 3738521 } })
80052025-04-30T06:13:23.204ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
80062025-04-30T06:13:23.205ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6f f7 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e o...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54100 req_id = 406d190b-1f62-40ae-a1a2-04156065fd78
80072025-04-30T06:13:23.205ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28663,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54100 req_id = 406d190b-1f62-40ae-a1a2-04156065fd78
80082025-04-30T06:13:23.206ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
80092025-04-30T06:13:23.206ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54100 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 406d190b-1f62-40ae-a1a2-04156065fd78
80102025-04-30T06:13:23.321ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
80112025-04-30T06:13:23.321ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
80122025-04-30T06:13:23.322ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
80132025-04-30T06:13:23.322ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80142025-04-30T06:13:23.322ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
80152025-04-30T06:13:23.323ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80162025-04-30T06:13:23.323ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: d9 57 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .W..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:53495 req_id = fa49d3c0-4fed-471c-a743-891208a08ca8
80172025-04-30T06:13:23.323ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55639,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53495 req_id = fa49d3c0-4fed-471c-a743-891208a08ca8
80182025-04-30T06:13:23.323ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
80192025-04-30T06:13:23.324ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 55639, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:53495 req_id = fa49d3c0-4fed-471c-a743-891208a08ca8
80202025-04-30T06:13:23.324ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
80212025-04-30T06:13:23.324ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
80222025-04-30T06:13:23.324ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
80232025-04-30T06:13:23.327ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "03b60d9c-8ff2-4ba9-8a38-ef046bca8681", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80242025-04-30T06:13:23.327ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
80252025-04-30T06:13:23.328ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
80262025-04-30T06:13:23.331ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a6b3774-107b-46de-926a-f9cb6123a608", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80272025-04-30T06:13:23.331ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
80282025-04-30T06:13:23.333ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7a14f26-203c-42f7-a867-a4f2c3d92163", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80292025-04-30T06:13:23.333ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
80302025-04-30T06:13:23.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "574fa590-34a7-4755-bd0e-60116d982ae3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80312025-04-30T06:13:23.336ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
80322025-04-30T06:13:23.336ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
80332025-04-30T06:13:23.336ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80342025-04-30T06:13:23.336ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
80352025-04-30T06:13:23.337ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80362025-04-30T06:13:23.338ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b9a66de-94f9-4e51-95d5-08088365f702", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80372025-04-30T06:13:23.339ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
80382025-04-30T06:13:23.341ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "345d37b7-3406-4f3a-b924-49222d3607e1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80392025-04-30T06:13:23.341ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
80402025-04-30T06:13:23.342ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
80412025-04-30T06:13:23.342ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
80422025-04-30T06:13:23.344ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
80432025-04-30T06:13:23.345ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "134eab51-c18b-4a47-aa3b-9f110ce3b125", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80442025-04-30T06:13:23.345ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
80452025-04-30T06:13:23.348ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "77e01429-1af0-4104-8c17-10f378b020ab", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80462025-04-30T06:13:23.348ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
80472025-04-30T06:13:23.349ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "250376ab-6090-41b7-845d-8cc5fcf0d4b4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80482025-04-30T06:13:23.349ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
80492025-04-30T06:13:23.350ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba12ae77-ea49-411e-8d6c-450b71bc7e33", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80502025-04-30T06:13:23.351ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
80512025-04-30T06:13:23.352ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa12e0ee-4b15-4a1d-8ba4-600d7813c10e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80522025-04-30T06:13:23.352ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
80532025-04-30T06:13:23.354ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19b824ee-c108-4a63-a8e9-7ac8b2e6e6f2", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80542025-04-30T06:13:23.354ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
80552025-04-30T06:13:23.355ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1cfeb41-0df2-49c1-8f5e-f7f2bea3373b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80562025-04-30T06:13:23.355ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
80572025-04-30T06:13:23.358ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9730315-1f56-448f-bbe1-85afbdc9d3e1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80582025-04-30T06:13:23.358ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
80592025-04-30T06:13:23.360ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ba0c3b6-bbf8-4585-bb9c-55d9967adf85", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80602025-04-30T06:13:23.360ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
80612025-04-30T06:13:23.361ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "49c5e8e4-5093-49d7-9857-fd56812cbb42", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80622025-04-30T06:13:23.362ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
80632025-04-30T06:13:23.363ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1080f33c-4b57-4958-ad76-f03b567fb613", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80642025-04-30T06:13:23.364ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
80652025-04-30T06:13:23.365ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af802d09-45a1-4a5d-ab8e-748d6652bb73", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80662025-04-30T06:13:23.365ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
80672025-04-30T06:13:23.367ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d85d55e5-6ba7-42b4-a1e1-35dc9e0a2ae9", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80682025-04-30T06:13:23.367ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
80692025-04-30T06:13:23.368ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ead8ce32-8338-4060-aa9f-435671ce4636", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80702025-04-30T06:13:23.368ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
80712025-04-30T06:13:23.370ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a044a24b-f724-4611-b203-c8a617a5fac0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80722025-04-30T06:13:23.370ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
80732025-04-30T06:13:23.372ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "40d92cb7-c60d-4bac-94a7-26bc0bc3c812", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80742025-04-30T06:13:23.372ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
80752025-04-30T06:13:23.374ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7643ef7f-9964-434b-8b2e-d563ad1192d4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80762025-04-30T06:13:23.374ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
80772025-04-30T06:13:23.375ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fe553a3-8a8e-4c69-824f-d5e85165dd64", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80782025-04-30T06:13:23.376ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
80792025-04-30T06:13:23.378ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "556434f5-afbf-4b85-80a1-bcb14b91b865", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80802025-04-30T06:13:23.378ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
80812025-04-30T06:13:23.380ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76664c13-a98a-427a-99b4-7a6ecbfc90b0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80822025-04-30T06:13:23.380ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
80832025-04-30T06:13:23.382ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bb79368-25a7-48aa-baa4-15688ff73f09", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80842025-04-30T06:13:23.382ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
80852025-04-30T06:13:23.385ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e999f2d2-9671-43ba-b30a-0d084d695922", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80862025-04-30T06:13:23.385ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
80872025-04-30T06:13:23.387ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f5d07f4-7559-4cc9-bc39-a94e4fa351ab", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80882025-04-30T06:13:23.387ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
80892025-04-30T06:13:23.389ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
80902025-04-30T06:13:23.390ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "05b36780-205d-4773-8438-253ca1926a85", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80912025-04-30T06:13:23.390ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
80922025-04-30T06:13:23.392ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "65193959-0c60-4490-b63e-dec646454848", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80932025-04-30T06:13:23.393ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
80942025-04-30T06:13:23.394ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1bc5e04-c498-4da9-9011-8c0021cd37bb", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80952025-04-30T06:13:23.395ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
80962025-04-30T06:13:23.398ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a8e798b-4b96-46a0-93b0-dcb227db08fe", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
80972025-04-30T06:13:23.398ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
80982025-04-30T06:13:23.403ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e59acf7-7345-40dd-bce5-0b527289bebf", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
80992025-04-30T06:13:23.404ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
81002025-04-30T06:13:23.407ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "883877b0-46b9-49b4-bf24-162d477c4238", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
81012025-04-30T06:13:23.407ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
81022025-04-30T06:13:23.416ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "72b5c69d-d3e4-4440-b56f-ae7dd7a0b0f6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81032025-04-30T06:13:23.417ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
81042025-04-30T06:13:23.425ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5a03fe6-64df-4232-a7a2-0a788f8e6225", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81052025-04-30T06:13:23.425ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
81062025-04-30T06:13:23.429ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c467f46-af92-40fc-9a6e-2aea70678603", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81072025-04-30T06:13:23.431ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
81082025-04-30T06:13:23.439ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa606f0a-c27f-4377-a931-5d96d07cc629", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81092025-04-30T06:13:23.439ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
81102025-04-30T06:13:23.441ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
81112025-04-30T06:13:23.441ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81122025-04-30T06:13:23.441ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81132025-04-30T06:13:23.441ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
81142025-04-30T06:13:23.442ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81152025-04-30T06:13:23.444ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85d24a86-2960-4fce-bb0a-336867aa9ea2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81162025-04-30T06:13:23.453ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
81172025-04-30T06:13:23.453ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db4253eb-353d-4db8-90e9-e6fed5356a3e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81182025-04-30T06:13:23.453ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
81192025-04-30T06:13:23.453ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e579fe4-d667-477e-85c6-1b1331ca63a7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81202025-04-30T06:13:23.453ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
81212025-04-30T06:13:23.453ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81fdf866-64ad-48db-9c97-e2c3665c8972", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81222025-04-30T06:13:23.453ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
81232025-04-30T06:13:23.453ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "47530de9-d378-4b83-8ad5-bbe549a0763b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81242025-04-30T06:13:23.453ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
81252025-04-30T06:13:23.454ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "637a8bce-eee8-42f7-b746-8806d01bd741", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
81262025-04-30T06:13:23.454ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
81272025-04-30T06:13:23.454ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81282025-04-30T06:13:23.454ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81292025-04-30T06:13:23.454ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81302025-04-30T06:13:23.456ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81312025-04-30T06:13:23.459ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81322025-04-30T06:13:23.466ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81332025-04-30T06:13:23.467ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
81342025-04-30T06:13:23.467ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81352025-04-30T06:13:23.467ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
81362025-04-30T06:13:23.467ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81372025-04-30T06:13:23.472ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
81382025-04-30T06:13:23.472ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
81392025-04-30T06:13:23.472ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81402025-04-30T06:13:23.472ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81412025-04-30T06:13:23.472ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81422025-04-30T06:13:23.472ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81432025-04-30T06:13:23.476ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81442025-04-30T06:13:23.476ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
81452025-04-30T06:13:23.477ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81462025-04-30T06:13:23.478ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81472025-04-30T06:13:23.484ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
81482025-04-30T06:13:23.484ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81492025-04-30T06:13:23.484ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81502025-04-30T06:13:23.484ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81512025-04-30T06:13:23.530ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
81522025-04-30T06:13:23.557ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
81532025-04-30T06:13:23.557ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
81542025-04-30T06:13:23.558ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:58861 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
81552025-04-30T06:13:23.558ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:58861 req_id = 39bea9ab-5597-48a7-aab5-599fe5a384fe sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
81562025-04-30T06:13:23.559ZINFOomicron-dev (dropshot): request completed latency_us = 608 local_addr = [::1]:61881 method = GET remote_addr = [::1]:58861 req_id = 39bea9ab-5597-48a7-aab5-599fe5a384fe response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
81572025-04-30T06:13:23.559ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "39bea9ab-5597-48a7-aab5-599fe5a384fe", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:22 GMT"} })
81582025-04-30T06:13:23.562ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
81592025-04-30T06:13:23.562ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
81602025-04-30T06:13:23.562ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
81612025-04-30T06:13:23.564ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
81622025-04-30T06:13:23.564ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81632025-04-30T06:13:23.564ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81642025-04-30T06:13:23.564ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81652025-04-30T06:13:23.565ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81662025-04-30T06:13:23.569ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81672025-04-30T06:13:23.573ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
81682025-04-30T06:13:23.573ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81692025-04-30T06:13:23.573ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81702025-04-30T06:13:23.573ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81712025-04-30T06:13:23.581ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
81722025-04-30T06:13:23.582ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81732025-04-30T06:13:23.582ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81742025-04-30T06:13:23.582ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
81752025-04-30T06:13:23.582ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81762025-04-30T06:13:23.585ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81772025-04-30T06:13:23.590ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
81782025-04-30T06:13:23.590ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81792025-04-30T06:13:23.590ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
81802025-04-30T06:13:23.592ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81812025-04-30T06:13:23.630ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
81822025-04-30T06:13:23.630ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
81832025-04-30T06:13:23.631ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
81842025-04-30T06:13:23.631ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
81852025-04-30T06:13:23.631ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
81862025-04-30T06:13:23.632ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 447.389725ms iteration = 26 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
81872025-04-30T06:13:23.632ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 27 reason = Signaled
81882025-04-30T06:13:23.632ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
81892025-04-30T06:13:23.632ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
81902025-04-30T06:13:23.632ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81912025-04-30T06:13:23.632ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81922025-04-30T06:13:23.634ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81932025-04-30T06:13:23.638ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
81942025-04-30T06:13:23.653ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
81952025-04-30T06:13:23.653ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
81962025-04-30T06:13:23.653ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
81972025-04-30T06:13:23.654ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
81982025-04-30T06:13:23.657ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
81992025-04-30T06:13:23.657ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
82002025-04-30T06:13:23.659ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 36 cf 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 6............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:57169 req_id = 34106d3a-8c63-4cce-beb0-dd5f3e5d962e
82012025-04-30T06:13:23.660ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14031,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57169 req_id = 34106d3a-8c63-4cce-beb0-dd5f3e5d962e
82022025-04-30T06:13:23.660ZDEBGomicron-dev (store): query key key = _mgs._tcp
82032025-04-30T06:13:23.660ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
82042025-04-30T06:13:23.660ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57169 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 34106d3a-8c63-4cce-beb0-dd5f3e5d962e
82052025-04-30T06:13:23.661ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2468, tv_nsec: 459828170 } })
82062025-04-30T06:13:23.661ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
82072025-04-30T06:13:23.661ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a0 ec 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51954 req_id = b6f6a518-fda9-4620-8f03-1722179b28b7
82082025-04-30T06:13:23.661ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41196,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51954 req_id = b6f6a518-fda9-4620-8f03-1722179b28b7
82092025-04-30T06:13:23.661ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
82102025-04-30T06:13:23.661ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51954 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = b6f6a518-fda9-4620-8f03-1722179b28b7
82112025-04-30T06:13:23.668ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
82122025-04-30T06:13:23.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
82132025-04-30T06:13:23.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
82142025-04-30T06:13:23.719ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
82152025-04-30T06:13:23.719ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
82162025-04-30T06:13:23.719ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82172025-04-30T06:13:23.719ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
82182025-04-30T06:13:23.721ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
82192025-04-30T06:13:23.721ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82202025-04-30T06:13:23.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
82212025-04-30T06:13:23.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
82222025-04-30T06:13:23.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
82232025-04-30T06:13:23.728ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
82242025-04-30T06:13:23.736ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
82252025-04-30T06:13:23.736ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
82262025-04-30T06:13:23.736ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82272025-04-30T06:13:23.736ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
82282025-04-30T06:13:23.737ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82292025-04-30T06:13:23.740ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 1f 7f 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50072 req_id = 91dcda9b-a42f-4645-afde-e32eb711b046
82302025-04-30T06:13:23.740ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8063,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50072 req_id = 91dcda9b-a42f-4645-afde-e32eb711b046
82312025-04-30T06:13:23.740ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
82322025-04-30T06:13:23.740ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 8063, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50072 req_id = 91dcda9b-a42f-4645-afde-e32eb711b046
82332025-04-30T06:13:23.741ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
82342025-04-30T06:13:23.741ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
82352025-04-30T06:13:23.741ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
82362025-04-30T06:13:23.742ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
82372025-04-30T06:13:23.742ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
82382025-04-30T06:13:23.743ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
82392025-04-30T06:13:23.744ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "87254d9a-a6dd-4af0-bb48-bdca3ab71358", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82402025-04-30T06:13:23.744ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
82412025-04-30T06:13:23.746ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d94e6ec6-e198-4f62-8b86-b8c0a9e04281", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82422025-04-30T06:13:23.746ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
82432025-04-30T06:13:23.748ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1e53630-f91b-438e-b696-0a8c619f15f0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82442025-04-30T06:13:23.748ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
82452025-04-30T06:13:23.751ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6eaf0102-8ebf-4e80-8ee0-8ae2ff4b19d0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82462025-04-30T06:13:23.751ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
82472025-04-30T06:13:23.753ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd73c46d-4249-40f0-9682-2adb033adb92", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82482025-04-30T06:13:23.753ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
82492025-04-30T06:13:23.756ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "63626221-ba2e-461a-9b48-184a59ee22e8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82502025-04-30T06:13:23.756ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
82512025-04-30T06:13:23.758ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c4909d5-4754-41c8-a6ac-f86cf09c98a1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82522025-04-30T06:13:23.759ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
82532025-04-30T06:13:23.761ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d837d3e-7f00-4e26-abe8-cac043f6fdc5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82542025-04-30T06:13:23.761ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
82552025-04-30T06:13:23.762ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a05a4f4c-0fe6-4983-a211-142435847200", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82562025-04-30T06:13:23.762ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
82572025-04-30T06:13:23.763ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "39a76c0a-9999-4298-8393-aa6d1dd48dba", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82582025-04-30T06:13:23.763ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
82592025-04-30T06:13:23.764ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "79592cc7-58e1-489f-9583-eb4819f4b54d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82602025-04-30T06:13:23.764ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
82612025-04-30T06:13:23.768ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e618e19-84a2-4ca4-b495-a8efe9093537", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82622025-04-30T06:13:23.769ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
82632025-04-30T06:13:23.770ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f6288bd-e7f5-4cac-a11a-d1f3ccdba15b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82642025-04-30T06:13:23.771ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
82652025-04-30T06:13:23.776ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "524d849c-3172-4cd4-a9b2-a33b775571a0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82662025-04-30T06:13:23.776ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
82672025-04-30T06:13:23.780ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d28f792-570e-4d9a-a0e0-8b4a9617de8c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82682025-04-30T06:13:23.780ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
82692025-04-30T06:13:23.783ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac193945-3e51-4892-bcc9-e6b7bbc35cd8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82702025-04-30T06:13:23.783ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
82712025-04-30T06:13:23.786ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa5aa731-6f36-41dd-95e7-cb66c8117e56", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82722025-04-30T06:13:23.786ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
82732025-04-30T06:13:23.787ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
82742025-04-30T06:13:23.796ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c00e3ed-3461-4f1c-8c1f-c05dcb2f8f61", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82752025-04-30T06:13:23.796ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
82762025-04-30T06:13:23.798ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bf7e5cb-d91b-40ac-b744-299fd6515f01", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82772025-04-30T06:13:23.798ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
82782025-04-30T06:13:23.799ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9e9cb42-e46b-4961-b8d8-690cf2f31a1d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82792025-04-30T06:13:23.799ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
82802025-04-30T06:13:23.800ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3464595e-4fea-4349-abce-0c50bca9b8b1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82812025-04-30T06:13:23.801ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
82822025-04-30T06:13:23.801ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3087ec24-94be-4da2-b2b7-eadca3ba08a2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82832025-04-30T06:13:23.802ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
82842025-04-30T06:13:23.804ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "93579d06-e0c2-4353-848b-503e886a71e4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82852025-04-30T06:13:23.804ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
82862025-04-30T06:13:23.815ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2684b2f-2a98-4110-8a35-e424af39730d", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82872025-04-30T06:13:23.815ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
82882025-04-30T06:13:23.819ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ededd890-d6f4-4f0f-86b9-5201f242ec6c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82892025-04-30T06:13:23.819ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
82902025-04-30T06:13:23.820ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55e1cfff-ed34-4447-a679-99e5c774c596", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82912025-04-30T06:13:23.821ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
82922025-04-30T06:13:23.825ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8372d510-7a5b-4d9f-a6d3-6b5f9056d8c6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82932025-04-30T06:13:23.826ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
82942025-04-30T06:13:23.829ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "37495f9d-f08d-4cac-878d-660fb449a202", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82952025-04-30T06:13:23.829ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
82962025-04-30T06:13:23.829ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "87706f9c-7af9-41a7-bba5-ef4e3c97504e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82972025-04-30T06:13:23.829ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
82982025-04-30T06:13:23.832ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee24e9ad-9807-493e-93a7-9e8e36a5e696", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
82992025-04-30T06:13:23.832ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
83002025-04-30T06:13:23.833ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8106ec0c-a7d4-4e23-b8c2-7b4afdcab55d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83012025-04-30T06:13:23.833ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
83022025-04-30T06:13:23.834ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbaf49fa-adda-4c46-8b1b-b178ccfc1b4a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83032025-04-30T06:13:23.835ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
83042025-04-30T06:13:23.836ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0270d54-ee4c-40e3-867a-d3402b0826e7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83052025-04-30T06:13:23.836ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
83062025-04-30T06:13:23.840ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
83072025-04-30T06:13:23.841ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83082025-04-30T06:13:23.841ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83092025-04-30T06:13:23.841ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
83102025-04-30T06:13:23.841ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83112025-04-30T06:13:23.844ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f5aaedf-8866-45de-ab30-935eb8a2cc0c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83122025-04-30T06:13:23.864ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
83132025-04-30T06:13:23.869ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83142025-04-30T06:13:23.870ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2605ca7-2caf-4548-8ed0-f6a527aaf3b3", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83152025-04-30T06:13:23.880ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
83162025-04-30T06:13:23.880ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83172025-04-30T06:13:23.880ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
83182025-04-30T06:13:23.882ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83192025-04-30T06:13:23.883ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
83202025-04-30T06:13:23.885ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83212025-04-30T06:13:23.886ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
83222025-04-30T06:13:23.889ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0fd341f-7da2-4d5c-946b-f1a62aa13d68", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83232025-04-30T06:13:23.889ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83242025-04-30T06:13:23.889ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
83252025-04-30T06:13:23.913ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "97f2e16c-1e4d-4b9a-b5a0-b9f24396077c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83262025-04-30T06:13:23.914ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
83272025-04-30T06:13:23.934ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
83282025-04-30T06:13:23.936ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "190941ac-0709-4bed-951e-740f43e9e8f5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83292025-04-30T06:13:23.937ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
83302025-04-30T06:13:23.940ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "97d60dda-973e-4e56-91f2-e58171b832ef", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83312025-04-30T06:13:23.940ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
83322025-04-30T06:13:23.942ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "543ce359-7889-427c-9b10-87e9631b65b6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83332025-04-30T06:13:23.942ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
83342025-04-30T06:13:23.944ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e928e482-3f35-43af-9d99-2f640c0e6fa7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83352025-04-30T06:13:23.944ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
83362025-04-30T06:13:23.945ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9cad0cf-14ad-4a23-b3d4-cf56b0145f4d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83372025-04-30T06:13:23.945ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
83382025-04-30T06:13:23.948ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e95f471-0cfe-4157-97cc-91d831e3d9fd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83392025-04-30T06:13:23.948ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
83402025-04-30T06:13:23.948ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce831257-d8d0-43a8-991c-181cb6d6c4a0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83412025-04-30T06:13:23.949ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
83422025-04-30T06:13:23.950ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02f8b784-b5f8-4620-8b60-1a94d5bf9215", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83432025-04-30T06:13:23.950ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
83442025-04-30T06:13:23.950ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83452025-04-30T06:13:23.950ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83462025-04-30T06:13:23.950ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83472025-04-30T06:13:23.951ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83482025-04-30T06:13:23.953ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83492025-04-30T06:13:23.965ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
83502025-04-30T06:13:23.965ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
83512025-04-30T06:13:23.965ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83522025-04-30T06:13:23.965ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83532025-04-30T06:13:23.965ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83542025-04-30T06:13:23.966ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83552025-04-30T06:13:23.975ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83562025-04-30T06:13:23.988ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
83572025-04-30T06:13:23.988ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83582025-04-30T06:13:23.988ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83592025-04-30T06:13:23.988ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
83602025-04-30T06:13:23.989ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83612025-04-30T06:13:23.999ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
83622025-04-30T06:13:23.999ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83632025-04-30T06:13:23.999ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83642025-04-30T06:13:23.999ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83652025-04-30T06:13:24.008ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83662025-04-30T06:13:24.019ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
83672025-04-30T06:13:24.019ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83682025-04-30T06:13:24.019ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
83692025-04-30T06:13:24.021ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83702025-04-30T06:13:24.050ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
83712025-04-30T06:13:24.050ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
83722025-04-30T06:13:24.051ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
83732025-04-30T06:13:24.067ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
83742025-04-30T06:13:24.067ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
83752025-04-30T06:13:24.068ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:59247 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
83762025-04-30T06:13:24.068ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:59247 req_id = 1bfcb99f-d7f0-41c4-acce-d42d294c6c92 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
83772025-04-30T06:13:24.069ZINFOomicron-dev (dropshot): request completed latency_us = 551 local_addr = [::1]:61881 method = GET remote_addr = [::1]:59247 req_id = 1bfcb99f-d7f0-41c4-acce-d42d294c6c92 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
83782025-04-30T06:13:24.069ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bfcb99f-d7f0-41c4-acce-d42d294c6c92", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
83792025-04-30T06:13:24.072ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
83802025-04-30T06:13:24.072ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
83812025-04-30T06:13:24.072ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
83822025-04-30T06:13:24.075ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
83832025-04-30T06:13:24.075ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
83842025-04-30T06:13:24.075ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83852025-04-30T06:13:24.075ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83862025-04-30T06:13:24.076ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83872025-04-30T06:13:24.078ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
83882025-04-30T06:13:24.082ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
83892025-04-30T06:13:24.083ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
83902025-04-30T06:13:24.083ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
83912025-04-30T06:13:24.083ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
83922025-04-30T06:13:24.109ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
83932025-04-30T06:13:24.109ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
83942025-04-30T06:13:24.110ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
83952025-04-30T06:13:24.111ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
83962025-04-30T06:13:24.116ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
83972025-04-30T06:13:24.116ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
83982025-04-30T06:13:24.126ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
83992025-04-30T06:13:24.126ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
84002025-04-30T06:13:24.129ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
84012025-04-30T06:13:24.142ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
84022025-04-30T06:13:24.143ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
84032025-04-30T06:13:24.143ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 510.958631ms iteration = 27 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
84042025-04-30T06:13:24.143ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 28 reason = Signaled
84052025-04-30T06:13:24.143ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
84062025-04-30T06:13:24.143ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
84072025-04-30T06:13:24.143ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
84082025-04-30T06:13:24.143ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84092025-04-30T06:13:24.144ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
84102025-04-30T06:13:24.161ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84112025-04-30T06:13:24.167ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
84122025-04-30T06:13:24.167ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
84132025-04-30T06:13:24.167ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
84142025-04-30T06:13:24.168ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
84152025-04-30T06:13:24.180ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
84162025-04-30T06:13:24.180ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
84172025-04-30T06:13:24.180ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84182025-04-30T06:13:24.180ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
84192025-04-30T06:13:24.181ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84202025-04-30T06:13:24.182ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
84212025-04-30T06:13:24.182ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
84222025-04-30T06:13:24.184ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 09 a3 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56649 req_id = 2f7581c3-d3d6-4973-b287-98485ffb8340
84232025-04-30T06:13:24.184ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2467,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56649 req_id = 2f7581c3-d3d6-4973-b287-98485ffb8340
84242025-04-30T06:13:24.184ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84252025-04-30T06:13:24.185ZDEBGomicron-dev (store): query key key = _mgs._tcp
84262025-04-30T06:13:24.185ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
84272025-04-30T06:13:24.186ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56649 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 2f7581c3-d3d6-4973-b287-98485ffb8340
84282025-04-30T06:13:24.186ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2468, tv_nsec: 985442933 } })
84292025-04-30T06:13:24.186ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
84302025-04-30T06:13:24.187ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 37 d5 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 7...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51247 req_id = e520da54-9808-4c88-8101-d514c6ab59e0
84312025-04-30T06:13:24.187ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 14293,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51247 req_id = e520da54-9808-4c88-8101-d514c6ab59e0
84322025-04-30T06:13:24.187ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
84332025-04-30T06:13:24.187ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51247 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e520da54-9808-4c88-8101-d514c6ab59e0
84342025-04-30T06:13:24.192ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
84352025-04-30T06:13:24.192ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84362025-04-30T06:13:24.192ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
84372025-04-30T06:13:24.193ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84382025-04-30T06:13:24.196ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84392025-04-30T06:13:24.196ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
84402025-04-30T06:13:24.196ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84412025-04-30T06:13:24.212ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
84422025-04-30T06:13:24.250ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
84432025-04-30T06:13:24.259ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
84442025-04-30T06:13:24.261ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: e0 b5 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62093 req_id = 8139b706-caa6-484e-8ab8-8613d7dc3440
84452025-04-30T06:13:24.261ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57525,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62093 req_id = 8139b706-caa6-484e-8ab8-8613d7dc3440
84462025-04-30T06:13:24.261ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
84472025-04-30T06:13:24.261ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57525, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62093 req_id = 8139b706-caa6-484e-8ab8-8613d7dc3440
84482025-04-30T06:13:24.261ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
84492025-04-30T06:13:24.261ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
84502025-04-30T06:13:24.261ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
84512025-04-30T06:13:24.265ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "470a37d6-c9f4-4e55-8060-ca86077e8c22", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84522025-04-30T06:13:24.265ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
84532025-04-30T06:13:24.267ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d3d7085-d57d-4eee-806b-d9f888b0bae4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84542025-04-30T06:13:24.267ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
84552025-04-30T06:13:24.274ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf49cceb-1f43-46ba-8c59-b32de3a529e8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84562025-04-30T06:13:24.276ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
84572025-04-30T06:13:24.280ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1c705fc-6fbb-4009-8424-59e6d6412e0e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84582025-04-30T06:13:24.280ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
84592025-04-30T06:13:24.283ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "401cb399-f1c9-4520-9c41-28380f09aca1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84602025-04-30T06:13:24.283ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
84612025-04-30T06:13:24.287ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "860430c2-59ea-433c-a7c1-fad491e1b81f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84622025-04-30T06:13:24.287ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
84632025-04-30T06:13:24.292ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "36deba14-eccc-4c20-a99c-97bbe41f7aad", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84642025-04-30T06:13:24.292ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
84652025-04-30T06:13:24.296ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6d32c92-cec2-41b4-9252-94c664c25dd1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84662025-04-30T06:13:24.296ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
84672025-04-30T06:13:24.299ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c8146d3-358c-4f5a-b89c-b65ff3a7b98e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84682025-04-30T06:13:24.299ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
84692025-04-30T06:13:24.308ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
84702025-04-30T06:13:24.308ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
84712025-04-30T06:13:24.308ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84722025-04-30T06:13:24.308ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
84732025-04-30T06:13:24.311ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84742025-04-30T06:13:24.313ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "930616c1-99b9-4afc-9bfd-102d0963700a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84752025-04-30T06:13:24.313ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
84762025-04-30T06:13:24.315ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
84772025-04-30T06:13:24.320ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
84782025-04-30T06:13:24.320ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84792025-04-30T06:13:24.320ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
84802025-04-30T06:13:24.320ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84812025-04-30T06:13:24.327ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
84822025-04-30T06:13:24.327ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
84832025-04-30T06:13:24.328ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
84842025-04-30T06:13:24.332ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86d8a7b8-16bf-4a7c-8cbe-4a460ce4d2e1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84852025-04-30T06:13:24.332ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
84862025-04-30T06:13:24.334ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "68206ef2-673b-4322-83e2-8a6b31f06654", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84872025-04-30T06:13:24.334ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
84882025-04-30T06:13:24.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef193696-120f-43cf-8add-a1a5f37897aa", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84892025-04-30T06:13:24.336ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
84902025-04-30T06:13:24.339ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d3eaa49-46bf-4b57-82ea-2868e4277242", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84912025-04-30T06:13:24.339ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
84922025-04-30T06:13:24.341ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3387854b-a588-4f5e-9724-f10539e7f4bb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84932025-04-30T06:13:24.341ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
84942025-04-30T06:13:24.343ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0703aab-1807-45f4-9ca5-16258bbc8948", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84952025-04-30T06:13:24.343ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
84962025-04-30T06:13:24.345ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a7db63c-b703-4cf9-a78c-f5fbd4e4818b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84972025-04-30T06:13:24.345ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
84982025-04-30T06:13:24.348ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9a3300d-63b5-40e2-b167-3ec3d954a18d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
84992025-04-30T06:13:24.349ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
85002025-04-30T06:13:24.352ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4fb40db-dfe4-4225-b357-81fdbd71eebd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85012025-04-30T06:13:24.352ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
85022025-04-30T06:13:24.353ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d2d231f-7dda-4db0-8c10-a1f52a93b049", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85032025-04-30T06:13:24.353ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
85042025-04-30T06:13:24.354ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9ef707d-3abc-4129-a568-517301b0dff7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85052025-04-30T06:13:24.354ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
85062025-04-30T06:13:24.356ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe1eac7a-d606-42c1-94d1-6472746c008f", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85072025-04-30T06:13:24.356ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
85082025-04-30T06:13:24.357ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf3f986f-1853-4f68-bd94-7b91295e9646", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85092025-04-30T06:13:24.357ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
85102025-04-30T06:13:24.358ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b626987d-2bbd-4550-a5c0-8669b77be532", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85112025-04-30T06:13:24.359ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
85122025-04-30T06:13:24.361ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "50af8d24-1b53-4eec-9ac9-9f3893acc274", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85132025-04-30T06:13:24.361ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
85142025-04-30T06:13:24.364ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db9e994c-0c14-4d2d-bad0-c25b90d18662", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85152025-04-30T06:13:24.364ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
85162025-04-30T06:13:24.366ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "32ca309b-6ea3-46cf-b7f9-f8366afe140c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85172025-04-30T06:13:24.366ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
85182025-04-30T06:13:24.368ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
85192025-04-30T06:13:24.423ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
85202025-04-30T06:13:24.423ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85212025-04-30T06:13:24.423ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85222025-04-30T06:13:24.423ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
85232025-04-30T06:13:24.424ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85242025-04-30T06:13:24.430ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85252025-04-30T06:13:24.437ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
85262025-04-30T06:13:24.438ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85272025-04-30T06:13:24.438ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
85282025-04-30T06:13:24.438ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85292025-04-30T06:13:24.440ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85302025-04-30T06:13:24.441ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
85312025-04-30T06:13:24.441ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85322025-04-30T06:13:24.467ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9049b3e-051c-4371-95c5-09d9f87df5da", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85332025-04-30T06:13:24.468ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
85342025-04-30T06:13:24.471ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8b6d324-366c-458d-be74-8fc2b5cfc208", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85352025-04-30T06:13:24.471ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
85362025-04-30T06:13:24.472ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
85372025-04-30T06:13:24.491ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d07fc83-09ba-401f-b56f-5bf4afb6aeb4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
85382025-04-30T06:13:24.491ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
85392025-04-30T06:13:24.496ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "760ed8f1-4d30-40f1-8c6e-1f2de53d5794", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85402025-04-30T06:13:24.496ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
85412025-04-30T06:13:24.498ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0859001-8c65-4251-8ab6-ec60bd71320e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85422025-04-30T06:13:24.498ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
85432025-04-30T06:13:24.501ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c56140b5-c48c-4176-b8f2-a674a83b675a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85442025-04-30T06:13:24.501ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
85452025-04-30T06:13:24.503ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf7a5420-2dfc-414c-a296-19e8e953b125", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
85462025-04-30T06:13:24.503ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
85472025-04-30T06:13:24.505ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6336052c-a6f0-4b3a-88fc-d719bc473a7c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
85482025-04-30T06:13:24.506ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
85492025-04-30T06:13:24.508ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5e78b06-84b4-4655-a448-b2d948d9eff9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85502025-04-30T06:13:24.509ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
85512025-04-30T06:13:24.511ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec16b317-8b34-4e46-a3f0-892a6bf2daec", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85522025-04-30T06:13:24.511ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
85532025-04-30T06:13:24.514ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f86067c-1646-44a9-9ab4-417efff226f2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85542025-04-30T06:13:24.514ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
85552025-04-30T06:13:24.516ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "644e070d-f9d7-43bd-b8d5-df3b58cec194", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85562025-04-30T06:13:24.516ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
85572025-04-30T06:13:24.518ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "17e290a0-80d0-42d1-be22-9874aa7370bd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85582025-04-30T06:13:24.518ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
85592025-04-30T06:13:24.520ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "67f18561-7e5a-4521-97a0-685a4e1f35e1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85602025-04-30T06:13:24.520ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
85612025-04-30T06:13:24.522ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03c93b42-970a-4a86-9934-722fe09ca207", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85622025-04-30T06:13:24.522ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
85632025-04-30T06:13:24.523ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
85642025-04-30T06:13:24.523ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85652025-04-30T06:13:24.524ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85662025-04-30T06:13:24.524ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
85672025-04-30T06:13:24.524ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85682025-04-30T06:13:24.525ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a58abd16-59ab-46a5-ad86-4aef0b7fb284", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
85692025-04-30T06:13:24.526ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
85702025-04-30T06:13:24.527ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85712025-04-30T06:13:24.529ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "028aca18-f632-4e5a-9f2b-a3d9161ff678", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85722025-04-30T06:13:24.529ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
85732025-04-30T06:13:24.530ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d18c67f3-9664-4ba7-b73c-e14d1cd498d1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
85742025-04-30T06:13:24.530ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
85752025-04-30T06:13:24.531ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85762025-04-30T06:13:24.531ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
85772025-04-30T06:13:24.531ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85782025-04-30T06:13:24.531ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
85792025-04-30T06:13:24.533ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
85802025-04-30T06:13:24.533ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85812025-04-30T06:13:24.533ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
85822025-04-30T06:13:24.533ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85832025-04-30T06:13:24.535ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85842025-04-30T06:13:24.535ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
85852025-04-30T06:13:24.535ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
85862025-04-30T06:13:24.536ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
85872025-04-30T06:13:24.539ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
85882025-04-30T06:13:24.539ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
85892025-04-30T06:13:24.539ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
85902025-04-30T06:13:24.539ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
85912025-04-30T06:13:24.539ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85922025-04-30T06:13:24.540ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
85932025-04-30T06:13:24.543ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
85942025-04-30T06:13:24.547ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
85952025-04-30T06:13:24.547ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
85962025-04-30T06:13:24.547ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
85972025-04-30T06:13:24.547ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
85982025-04-30T06:13:24.571ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
85992025-04-30T06:13:24.621ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
86002025-04-30T06:13:24.621ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
86012025-04-30T06:13:24.622ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:55266 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
86022025-04-30T06:13:24.622ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:55266 req_id = 6f7e33c2-e87b-4a55-9657-96d8a989c396 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
86032025-04-30T06:13:24.623ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
86042025-04-30T06:13:24.623ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86052025-04-30T06:13:24.623ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86062025-04-30T06:13:24.623ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
86072025-04-30T06:13:24.623ZINFOomicron-dev (dropshot): request completed latency_us = 625 local_addr = [::1]:61881 method = GET remote_addr = [::1]:55266 req_id = 6f7e33c2-e87b-4a55-9657-96d8a989c396 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
86082025-04-30T06:13:24.623ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f7e33c2-e87b-4a55-9657-96d8a989c396", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
86092025-04-30T06:13:24.624ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86102025-04-30T06:13:24.625ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
86112025-04-30T06:13:24.625ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
86122025-04-30T06:13:24.625ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
86132025-04-30T06:13:24.627ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
86142025-04-30T06:13:24.627ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86152025-04-30T06:13:24.627ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86162025-04-30T06:13:24.627ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86172025-04-30T06:13:24.628ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86182025-04-30T06:13:24.635ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86192025-04-30T06:13:24.637ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86202025-04-30T06:13:24.641ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
86212025-04-30T06:13:24.641ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86222025-04-30T06:13:24.641ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
86232025-04-30T06:13:24.642ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86242025-04-30T06:13:24.642ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
86252025-04-30T06:13:24.642ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86262025-04-30T06:13:24.642ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86272025-04-30T06:13:24.643ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86282025-04-30T06:13:24.653ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86292025-04-30T06:13:24.653ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
86302025-04-30T06:13:24.653ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86312025-04-30T06:13:24.677ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
86322025-04-30T06:13:24.677ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
86332025-04-30T06:13:24.677ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 534.249567ms iteration = 28 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
86342025-04-30T06:13:24.677ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 29 reason = Signaled
86352025-04-30T06:13:24.677ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
86362025-04-30T06:13:24.677ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86372025-04-30T06:13:24.677ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86382025-04-30T06:13:24.677ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86392025-04-30T06:13:24.678ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86402025-04-30T06:13:24.680ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
86412025-04-30T06:13:24.680ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
86422025-04-30T06:13:24.681ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = d2f3da49-36fe-4b3c-87be-5080b9539105 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
86432025-04-30T06:13:24.681ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = d2f3da49-36fe-4b3c-87be-5080b9539105 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
86442025-04-30T06:13:24.681ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = d2f3da49-36fe-4b3c-87be-5080b9539105 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
86452025-04-30T06:13:24.681ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
86462025-04-30T06:13:24.682ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = d2f3da49-36fe-4b3c-87be-5080b9539105 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
86472025-04-30T06:13:24.682ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
86482025-04-30T06:13:24.687ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
86492025-04-30T06:13:24.687ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
86502025-04-30T06:13:24.687ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
86512025-04-30T06:13:24.688ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
86522025-04-30T06:13:24.697ZINFOomicron-dev (dropshot_internal): request completed latency_us = 16387 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = d2f3da49-36fe-4b3c-87be-5080b9539105 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
86532025-04-30T06:13:24.697ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2f3da49-36fe-4b3c-87be-5080b9539105", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
86542025-04-30T06:13:24.698ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
86552025-04-30T06:13:24.698ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
86562025-04-30T06:13:24.698ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
86572025-04-30T06:13:24.699ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 93e5dcae-14d3-4199-b81b-77d7045ee626 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
86582025-04-30T06:13:24.700ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 93e5dcae-14d3-4199-b81b-77d7045ee626 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
86592025-04-30T06:13:24.701ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 93e5dcae-14d3-4199-b81b-77d7045ee626 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
86602025-04-30T06:13:24.701ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 2d 60 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 -`..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61937 req_id = fb3387b7-aeef-4829-96c2-7472666f2bc1
86612025-04-30T06:13:24.701ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11616,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61937 req_id = fb3387b7-aeef-4829-96c2-7472666f2bc1
86622025-04-30T06:13:24.702ZDEBGomicron-dev (store): query key key = _mgs._tcp
86632025-04-30T06:13:24.702ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
86642025-04-30T06:13:24.702ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61937 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = fb3387b7-aeef-4829-96c2-7472666f2bc1
86652025-04-30T06:13:24.702ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 93e5dcae-14d3-4199-b81b-77d7045ee626 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
86662025-04-30T06:13:24.703ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2469, tv_nsec: 501987464 } })
86672025-04-30T06:13:24.703ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
86682025-04-30T06:13:24.706ZINFOomicron-dev (dropshot_internal): request completed latency_us = 6223 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 93e5dcae-14d3-4199-b81b-77d7045ee626 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
86692025-04-30T06:13:24.707ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "93e5dcae-14d3-4199-b81b-77d7045ee626", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:23 GMT"} })
86702025-04-30T06:13:24.708ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
86712025-04-30T06:13:24.708ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
86722025-04-30T06:13:24.708ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
86732025-04-30T06:13:24.708ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 4f 56 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e OV..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60465 req_id = 0980b953-e07a-482c-9a46-071d15707e02
86742025-04-30T06:13:24.708ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20310,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60465 req_id = 0980b953-e07a-482c-9a46-071d15707e02
86752025-04-30T06:13:24.708ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
86762025-04-30T06:13:24.708ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60465 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0980b953-e07a-482c-9a46-071d15707e02
86772025-04-30T06:13:24.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
86782025-04-30T06:13:24.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
86792025-04-30T06:13:24.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
86802025-04-30T06:13:24.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
86812025-04-30T06:13:24.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
86822025-04-30T06:13:24.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
86832025-04-30T06:13:24.721ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
86842025-04-30T06:13:24.721ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
86852025-04-30T06:13:24.722ZINFOomicron-dev (dropshot): accepted connection file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:1025 local_addr = [::1]:50290 remote_addr = [::1]:62154
86862025-04-30T06:13:24.722ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:50290 method = GET remote_addr = [::1]:62154 req_id = 70bbc4a4-4ece-4b5b-9351-1e272e835e4a uri = /e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
86872025-04-30T06:13:24.726ZINFOomicron-dev (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 4323 local_addr = [::1]:50290 method = GET remote_addr = [::1]:62154 req_id = 70bbc4a4-4ece-4b5b-9351-1e272e835e4a response_code = 200 uri = /e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
86882025-04-30T06:13:24.727ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
86892025-04-30T06:13:24.734ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
86902025-04-30T06:13:24.734ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
86912025-04-30T06:13:24.734ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
86922025-04-30T06:13:24.734ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
86932025-04-30T06:13:24.735ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
86942025-04-30T06:13:24.752ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
86952025-04-30T06:13:24.752ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 5 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
86962025-04-30T06:13:24.752ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
86972025-04-30T06:13:24.752ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
86982025-04-30T06:13:24.752ZDEBGomicron-dev (results-sink-cluster): inserting 18 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
86992025-04-30T06:13:24.752ZDEBGomicron-dev (results-sink): inserting 18 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
87002025-04-30T06:13:24.752ZDEBGomicron-dev (clickhouse-client): unrolling 18 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87012025-04-30T06:13:24.752ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 2e 80 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:49890 req_id = f832a09f-a378-4fe1-9f0b-6913f6ae1d51
87022025-04-30T06:13:24.752ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11904,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49890 req_id = f832a09f-a378-4fe1-9f0b-6913f6ae1d51
87032025-04-30T06:13:24.753ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
87042025-04-30T06:13:24.753ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11904, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:49890 req_id = f832a09f-a378-4fe1-9f0b-6913f6ae1d51
87052025-04-30T06:13:24.753ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
87062025-04-30T06:13:24.753ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
87072025-04-30T06:13:24.753ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
87082025-04-30T06:13:24.754ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
87092025-04-30T06:13:24.756ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5310d688-31c9-4875-a56b-a5b1eb623f46", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87102025-04-30T06:13:24.756ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
87112025-04-30T06:13:24.757ZDEBGomicron-dev (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87122025-04-30T06:13:24.757ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('integration_target:integration_metric') FORMAT Native;
87132025-04-30T06:13:24.758ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5de1571b-ddd8-44aa-ab6b-10a1693ffe6a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87142025-04-30T06:13:24.758ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
87152025-04-30T06:13:24.760ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
87162025-04-30T06:13:24.760ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
87172025-04-30T06:13:24.760ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
87182025-04-30T06:13:24.761ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87192025-04-30T06:13:24.761ZTRAComicron-dev (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87202025-04-30T06:13:24.762ZDEBGomicron-dev (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:13:24.757537736Z } timeseries_name = http_service:request_latency_histogram
87212025-04-30T06:13:24.762ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c613e35f-71d7-49a3-b519-59a327596ab3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87222025-04-30T06:13:24.762ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
87232025-04-30T06:13:24.763ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
87242025-04-30T06:13:24.763ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
87252025-04-30T06:13:24.764ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
87262025-04-30T06:13:24.765ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "14639da5-42d2-49ca-9d7a-16dec5040aff", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87272025-04-30T06:13:24.766ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
87282025-04-30T06:13:24.769ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "07a0297a-2ef4-4516-93dc-886e1306f653", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87292025-04-30T06:13:24.769ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
87302025-04-30T06:13:24.770ZDEBGomicron-dev (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87312025-04-30T06:13:24.770ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
87322025-04-30T06:13:24.774ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b24c4090-457c-46d8-83cf-833f8c9fe6b6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87332025-04-30T06:13:24.775ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
87342025-04-30T06:13:24.775ZTRAComicron-dev (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87352025-04-30T06:13:24.775ZDEBGomicron-dev (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:cpus_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:13:24.769982422Z } timeseries_name = collection_target:cpus_provisioned
87362025-04-30T06:13:24.777ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cc8d27c-01ae-4a7e-84ee-591ebbfdef6f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87372025-04-30T06:13:24.777ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
87382025-04-30T06:13:24.777ZDEBGomicron-dev (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87392025-04-30T06:13:24.777ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('collection_target:cpus_provisioned', 'http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
87402025-04-30T06:13:24.779ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d93d99f-e20d-44da-9156-fbbcb44c8941", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87412025-04-30T06:13:24.779ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
87422025-04-30T06:13:24.780ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8878662-9c56-4253-8fdc-155695d4dc64", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87432025-04-30T06:13:24.780ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
87442025-04-30T06:13:24.781ZTRAComicron-dev (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87452025-04-30T06:13:24.781ZDEBGomicron-dev (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:ram_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:13:24.777301805Z } timeseries_name = collection_target:ram_provisioned
87462025-04-30T06:13:24.782ZDEBGomicron-dev (clickhouse-client): retrieving timeseries schema from database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87472025-04-30T06:13:24.782ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2c7d7fd-e4f9-4987-b79d-646a56773edf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87482025-04-30T06:13:24.782ZTRAComicron-dev (clickhouse-client): executing SQL query collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e sql = SELECT * FROM oximeter.timeseries_schema WHERE timeseries_name NOT IN ('collection_target:cpus_provisioned', 'collection_target:ram_provisioned', 'http_service:request_latency_histogram', 'integration_target:integration_metric') FORMAT Native;
87492025-04-30T06:13:24.782ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
87502025-04-30T06:13:24.783ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "99c7a300-d41a-404c-bbb6-30a73f90403d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87512025-04-30T06:13:24.783ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
87522025-04-30T06:13:24.784ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e10a7a5b-5789-4bfa-b79b-01b90b95fe78", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87532025-04-30T06:13:24.784ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
87542025-04-30T06:13:24.785ZTRAComicron-dev (clickhouse-client): no new timeseries schema in database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87552025-04-30T06:13:24.785ZDEBGomicron-dev (clickhouse-client): new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:virtual_disk_space_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2025-04-30T06:13:24.782053773Z } timeseries_name = collection_target:virtual_disk_space_provisioned
87562025-04-30T06:13:24.786ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "65bad64f-688b-48c3-8f1f-aa961237d403", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87572025-04-30T06:13:24.786ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
87582025-04-30T06:13:24.791ZDEBGomicron-dev (clickhouse-client): inserting 4 new timeseries schema collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
87592025-04-30T06:13:24.791ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 6 n_rows = 4 sql = INSERT INTO oximeter.timeseries_schema FORMAT Native
87602025-04-30T06:13:24.795ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 12 sql = INSERT INTO oximeter.fields_string FORMAT Native
87612025-04-30T06:13:24.799ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0244c8c-d4ce-4343-97dd-125a31470942", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87622025-04-30T06:13:24.799ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 12 table_name = fields_string
87632025-04-30T06:13:24.799ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 5 sql = INSERT INTO oximeter.fields_u16 FORMAT Native
87642025-04-30T06:13:24.799ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
87652025-04-30T06:13:24.803ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 5 table_name = fields_u16
87662025-04-30T06:13:24.803ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 17 sql = INSERT INTO oximeter.fields_uuid FORMAT Native
87672025-04-30T06:13:24.805ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f931b52c-eedd-41de-bce5-729531f1e04d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87682025-04-30T06:13:24.805ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
87692025-04-30T06:13:24.808ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 17 table_name = fields_uuid
87702025-04-30T06:13:24.808ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 19 n_rows = 5 sql = INSERT INTO oximeter.measurements_histogramu64 FORMAT Native
87712025-04-30T06:13:24.813ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8157bbf4-e352-456c-a95f-f1bd07923491", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87722025-04-30T06:13:24.813ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
87732025-04-30T06:13:24.819ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf1a4235-8098-4aeb-8452-b63d967ba08d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87742025-04-30T06:13:24.819ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
87752025-04-30T06:13:24.819ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "de2f2922-0fb5-401b-8fbe-bb1e1ab545f7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87762025-04-30T06:13:24.819ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
87772025-04-30T06:13:24.821ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e88189b9-58e9-4185-87e2-b72b06477188", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87782025-04-30T06:13:24.821ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
87792025-04-30T06:13:24.822ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c25c5998-0e13-4762-b34b-5ea5801d66df", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87802025-04-30T06:13:24.822ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
87812025-04-30T06:13:24.825ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c283919-7e1a-44fd-aaae-c5b4ba6b4691", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87822025-04-30T06:13:24.861ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
87832025-04-30T06:13:24.861ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de7a4d50-90ca-42a7-8155-64f57c873a8d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87842025-04-30T06:13:24.861ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
87852025-04-30T06:13:24.862ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b41ad36c-c2f1-4c3c-ae9c-a2a4c45f8044", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87862025-04-30T06:13:24.862ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
87872025-04-30T06:13:24.862ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8dccf0a6-54b0-4a5c-aa20-18b62e9ab033", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87882025-04-30T06:13:24.862ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
87892025-04-30T06:13:24.862ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
87902025-04-30T06:13:24.862ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "88f9ce03-159c-4e55-82f7-be1290c9e786", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87912025-04-30T06:13:24.862ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
87922025-04-30T06:13:24.862ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf7d222e-87fc-46f8-a9eb-e3d72fcee501", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87932025-04-30T06:13:24.863ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
87942025-04-30T06:13:24.863ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e7b15ae-a8db-4f4e-be26-42eec8c66bad", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87952025-04-30T06:13:24.863ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
87962025-04-30T06:13:24.863ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 5 table_name = measurements_histogramu64
87972025-04-30T06:13:24.863ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 13 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
87982025-04-30T06:13:24.863ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbfea68a-00bb-4e59-8057-655e49f49c4f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
87992025-04-30T06:13:24.863ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 13 table_name = measurements_i64
88002025-04-30T06:13:24.863ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
88012025-04-30T06:13:24.863ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
88022025-04-30T06:13:24.864ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
88032025-04-30T06:13:24.864ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "705eb176-cbf0-4067-b200-88fe15da947d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88042025-04-30T06:13:24.864ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
88052025-04-30T06:13:24.864ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46a467cf-2f71-4c8b-9f7d-791733a64f4d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88062025-04-30T06:13:24.864ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
88072025-04-30T06:13:24.866ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f48840c7-a4cd-47eb-85c7-d46e68e1b28c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88082025-04-30T06:13:24.866ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
88092025-04-30T06:13:24.868ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a06f5d0-b71d-4294-9f21-0003c5b0a654", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88102025-04-30T06:13:24.868ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
88112025-04-30T06:13:24.872ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1daceefa-6040-47c0-9c05-0c4d6bca03ee", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88122025-04-30T06:13:24.872ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
88132025-04-30T06:13:24.873ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "04f502e8-6403-42ca-a821-3669236eef82", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88142025-04-30T06:13:24.873ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
88152025-04-30T06:13:24.874ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0852a4e7-44bd-4381-b68c-6e5381459494", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88162025-04-30T06:13:24.874ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
88172025-04-30T06:13:24.876ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "be251329-e49c-4de1-8c73-d7845d984648", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88182025-04-30T06:13:24.876ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
88192025-04-30T06:13:24.880ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
88202025-04-30T06:13:24.880ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88212025-04-30T06:13:24.880ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88222025-04-30T06:13:24.880ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
88232025-04-30T06:13:24.882ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88242025-04-30T06:13:24.888ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3256577-65df-4af5-bddf-4133aceb2b59", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88252025-04-30T06:13:24.889ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
88262025-04-30T06:13:24.891ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35cc3776-a96b-4dc6-8241-c39d33fac8a4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88272025-04-30T06:13:24.891ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
88282025-04-30T06:13:24.893ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e3deb2c-50e9-4276-b604-017ef57f3c3a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88292025-04-30T06:13:24.893ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
88302025-04-30T06:13:24.895ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d991a1db-9078-4f8f-9075-e279daaaa80b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88312025-04-30T06:13:24.895ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
88322025-04-30T06:13:24.897ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "69368178-cc95-4da0-930b-f77950d317bf", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88332025-04-30T06:13:24.897ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
88342025-04-30T06:13:24.899ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "10a014d9-e823-4dc7-96d3-51a7e3a9b41c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88352025-04-30T06:13:24.899ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
88362025-04-30T06:13:24.901ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9466933f-9086-4f58-87e2-3f7889ae8eee", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88372025-04-30T06:13:24.901ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
88382025-04-30T06:13:24.902ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "084bbeef-2903-4bea-99ef-178a2232b59d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88392025-04-30T06:13:24.902ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
88402025-04-30T06:13:24.903ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2079c61-d9d7-4785-996a-f0ef031e6cb6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88412025-04-30T06:13:24.903ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
88422025-04-30T06:13:24.904ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88432025-04-30T06:13:24.904ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88442025-04-30T06:13:24.904ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88452025-04-30T06:13:24.904ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88462025-04-30T06:13:24.905ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88472025-04-30T06:13:24.909ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88482025-04-30T06:13:24.914ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
88492025-04-30T06:13:24.914ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
88502025-04-30T06:13:24.918ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
88512025-04-30T06:13:24.919ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88522025-04-30T06:13:24.919ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88532025-04-30T06:13:24.919ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88542025-04-30T06:13:24.920ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88552025-04-30T06:13:24.920ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88562025-04-30T06:13:24.920ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
88572025-04-30T06:13:24.921ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88582025-04-30T06:13:24.924ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88592025-04-30T06:13:24.932ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
88602025-04-30T06:13:24.933ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88612025-04-30T06:13:24.933ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88622025-04-30T06:13:24.933ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
88632025-04-30T06:13:24.933ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88642025-04-30T06:13:24.934ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88652025-04-30T06:13:24.935ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88662025-04-30T06:13:25.033ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
88672025-04-30T06:13:25.070ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
88682025-04-30T06:13:25.070ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
88692025-04-30T06:13:25.071ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:53417 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
88702025-04-30T06:13:25.072ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:53417 req_id = 87deee34-55a3-4b2c-a5fe-392df55f50fb sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
88712025-04-30T06:13:25.072ZINFOomicron-dev (dropshot): request completed latency_us = 564 local_addr = [::1]:61881 method = GET remote_addr = [::1]:53417 req_id = 87deee34-55a3-4b2c-a5fe-392df55f50fb response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
88722025-04-30T06:13:25.073ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "87deee34-55a3-4b2c-a5fe-392df55f50fb", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
88732025-04-30T06:13:25.075ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
88742025-04-30T06:13:25.075ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
88752025-04-30T06:13:25.075ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
88762025-04-30T06:13:25.077ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
88772025-04-30T06:13:25.077ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88782025-04-30T06:13:25.077ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88792025-04-30T06:13:25.078ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88802025-04-30T06:13:25.078ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88812025-04-30T06:13:25.083ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88822025-04-30T06:13:25.089ZTRAComicron-dev (results-sink): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
88832025-04-30T06:13:25.090ZTRAComicron-dev (results-sink-cluster): batch interval expired, but no samples to insert collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
88842025-04-30T06:13:25.094ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
88852025-04-30T06:13:25.094ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
88862025-04-30T06:13:25.094ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88872025-04-30T06:13:25.094ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
88882025-04-30T06:13:25.095ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
88892025-04-30T06:13:25.095ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
88902025-04-30T06:13:25.095ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
88912025-04-30T06:13:25.095ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88922025-04-30T06:13:25.096ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
88932025-04-30T06:13:25.118ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
88942025-04-30T06:13:25.131ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
88952025-04-30T06:13:25.131ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88962025-04-30T06:13:25.132ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
88972025-04-30T06:13:25.132ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
88982025-04-30T06:13:25.155ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
88992025-04-30T06:13:25.155ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
89002025-04-30T06:13:25.156ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
89012025-04-30T06:13:25.157ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
89022025-04-30T06:13:25.157ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 480.3473ms iteration = 29 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
89032025-04-30T06:13:25.157ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 30 reason = Signaled
89042025-04-30T06:13:25.157ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
89052025-04-30T06:13:25.158ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
89062025-04-30T06:13:25.158ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
89072025-04-30T06:13:25.158ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
89082025-04-30T06:13:25.159ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89092025-04-30T06:13:25.159ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
89102025-04-30T06:13:25.177ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89112025-04-30T06:13:25.183ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
89122025-04-30T06:13:25.184ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
89132025-04-30T06:13:25.184ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
89142025-04-30T06:13:25.187ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
89152025-04-30T06:13:25.206ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
89162025-04-30T06:13:25.207ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
89172025-04-30T06:13:25.208ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b7 23 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .#..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63821 req_id = badbd5ce-d168-46bd-ab20-2d043a152a99
89182025-04-30T06:13:25.208ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46883,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63821 req_id = badbd5ce-d168-46bd-ab20-2d043a152a99
89192025-04-30T06:13:25.208ZDEBGomicron-dev (store): query key key = _mgs._tcp
89202025-04-30T06:13:25.208ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
89212025-04-30T06:13:25.208ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63821 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = badbd5ce-d168-46bd-ab20-2d043a152a99
89222025-04-30T06:13:25.209ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2470, tv_nsec: 7719612 } })
89232025-04-30T06:13:25.209ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
89242025-04-30T06:13:25.210ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e9 03 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58257 req_id = 1c78e728-4809-4484-ab2e-efb277c214d2
89252025-04-30T06:13:25.210ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59651,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58257 req_id = 1c78e728-4809-4484-ab2e-efb277c214d2
89262025-04-30T06:13:25.211ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
89272025-04-30T06:13:25.212ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58257 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1c78e728-4809-4484-ab2e-efb277c214d2
89282025-04-30T06:13:25.233ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
89292025-04-30T06:13:25.286ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
89302025-04-30T06:13:25.286ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
89312025-04-30T06:13:25.286ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89322025-04-30T06:13:25.286ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
89332025-04-30T06:13:25.287ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89342025-04-30T06:13:25.287ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
89352025-04-30T06:13:25.288ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: b0 e9 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51690 req_id = 51541cde-58db-4471-8db4-fd9a8d1e15be
89362025-04-30T06:13:25.288ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 45289,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51690 req_id = 51541cde-58db-4471-8db4-fd9a8d1e15be
89372025-04-30T06:13:25.289ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
89382025-04-30T06:13:25.289ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 45289, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51690 req_id = 51541cde-58db-4471-8db4-fd9a8d1e15be
89392025-04-30T06:13:25.290ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
89402025-04-30T06:13:25.290ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
89412025-04-30T06:13:25.290ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
89422025-04-30T06:13:25.292ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
89432025-04-30T06:13:25.293ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d8c6e58-5417-4deb-97dc-c50b4115546e", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89442025-04-30T06:13:25.293ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
89452025-04-30T06:13:25.295ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6015dd97-df92-4ed4-9a57-481d9e5fb68b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89462025-04-30T06:13:25.295ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
89472025-04-30T06:13:25.298ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
89482025-04-30T06:13:25.298ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89492025-04-30T06:13:25.298ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
89502025-04-30T06:13:25.298ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89512025-04-30T06:13:25.299ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "183564bb-c674-4467-8d71-23a3d742db79", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89522025-04-30T06:13:25.299ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
89532025-04-30T06:13:25.301ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
89542025-04-30T06:13:25.301ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
89552025-04-30T06:13:25.302ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8b07fab-39aa-489a-a79b-53b2cdf7c688", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89562025-04-30T06:13:25.302ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
89572025-04-30T06:13:25.302ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
89582025-04-30T06:13:25.305ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16b45c08-a68b-4ac8-bbfb-5907c755d882", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89592025-04-30T06:13:25.305ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
89602025-04-30T06:13:25.308ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f0f7f84-7861-4c18-bea5-b1ce0b15cfb3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89612025-04-30T06:13:25.308ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
89622025-04-30T06:13:25.310ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "17b7a436-f40f-40ac-9050-697daa63abab", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89632025-04-30T06:13:25.310ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
89642025-04-30T06:13:25.313ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffe16db9-03ee-4cdc-8365-dc427c21b51c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89652025-04-30T06:13:25.313ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
89662025-04-30T06:13:25.314ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3bc0d00-9432-4ef2-a39d-42f04f7ec4db", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89672025-04-30T06:13:25.315ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
89682025-04-30T06:13:25.322ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaa312a6-10e5-484b-ad52-0842de5f2092", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89692025-04-30T06:13:25.322ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
89702025-04-30T06:13:25.323ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce032691-ad2b-4205-9f2f-d4f8a4f4f8a0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89712025-04-30T06:13:25.323ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
89722025-04-30T06:13:25.324ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "92154ec8-2a02-4928-87dc-e01acc0d5f36", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89732025-04-30T06:13:25.325ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
89742025-04-30T06:13:25.326ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "09e610a6-3ee3-4e88-bc80-09e4f68b052f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89752025-04-30T06:13:25.327ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
89762025-04-30T06:13:25.329ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbd32b86-97b6-4026-95b0-ea304decf86c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89772025-04-30T06:13:25.329ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
89782025-04-30T06:13:25.331ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bae39a20-a873-42b1-875f-4bac99124a79", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89792025-04-30T06:13:25.332ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
89802025-04-30T06:13:25.334ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f229ef26-a405-48f8-b219-72e12d65593d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89812025-04-30T06:13:25.334ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
89822025-04-30T06:13:25.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "725f910a-05a7-4718-9632-f9bd8f6ee4ee", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89832025-04-30T06:13:25.336ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
89842025-04-30T06:13:25.338ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41988c84-08f2-4a3e-812b-ffbaacdee635", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89852025-04-30T06:13:25.338ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
89862025-04-30T06:13:25.340ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
89872025-04-30T06:13:25.340ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6f87e0f-c087-4fb6-a03a-10ff8cd4f0e5", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89882025-04-30T06:13:25.340ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
89892025-04-30T06:13:25.342ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e05326fa-81c4-4658-847d-d7a47fb949af", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89902025-04-30T06:13:25.342ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
89912025-04-30T06:13:25.343ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c05dd9a-47a7-4ce0-9eef-ce1958dbfca4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89922025-04-30T06:13:25.343ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
89932025-04-30T06:13:25.345ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "52942247-429a-49a8-9513-74f5a8360c54", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89942025-04-30T06:13:25.345ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
89952025-04-30T06:13:25.346ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2db7687c-2fbd-4d64-9a47-4550fec2df3c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89962025-04-30T06:13:25.346ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
89972025-04-30T06:13:25.347ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "70268320-2e72-47d2-82da-69a3dce1192a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
89982025-04-30T06:13:25.348ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
89992025-04-30T06:13:25.349ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0618c87-f78f-484f-b1be-c5e230484a60", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90002025-04-30T06:13:25.349ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
90012025-04-30T06:13:25.352ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b659d52b-a052-450d-ba8c-f409e6fc1c03", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90022025-04-30T06:13:25.352ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
90032025-04-30T06:13:25.362ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea05b409-211d-423d-8432-e9697d55ffb2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90042025-04-30T06:13:25.362ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
90052025-04-30T06:13:25.366ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d82e82d-1cff-4c62-ad8a-341b82e09174", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90062025-04-30T06:13:25.367ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
90072025-04-30T06:13:25.370ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6be2ad2a-18a1-4a86-9413-6dd97b0ebdd9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90082025-04-30T06:13:25.370ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
90092025-04-30T06:13:25.373ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4db5135-26e3-415f-a0d1-85403227f77c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90102025-04-30T06:13:25.373ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
90112025-04-30T06:13:25.376ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c00aeae-f6d8-4df1-adfd-ac6409eec50f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90122025-04-30T06:13:25.377ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
90132025-04-30T06:13:25.388ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b468606b-8e00-4c4c-92ca-a9cd17f3fea5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90142025-04-30T06:13:25.389ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
90152025-04-30T06:13:25.390ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cf5891d-9e59-4ffb-88c7-a12a0b58dac6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90162025-04-30T06:13:25.391ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
90172025-04-30T06:13:25.392ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
90182025-04-30T06:13:25.393ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90192025-04-30T06:13:25.393ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90202025-04-30T06:13:25.393ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
90212025-04-30T06:13:25.398ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90222025-04-30T06:13:25.403ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90232025-04-30T06:13:25.410ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "277c9823-4798-4afd-a9ef-fed8a7d8ad24", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90242025-04-30T06:13:25.410ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
90252025-04-30T06:13:25.415ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
90262025-04-30T06:13:25.415ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90272025-04-30T06:13:25.415ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
90282025-04-30T06:13:25.416ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfbc2194-bdb5-48bb-b1f7-190f1d2664e7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90292025-04-30T06:13:25.416ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
90302025-04-30T06:13:25.417ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90312025-04-30T06:13:25.420ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c542d55-2b49-476b-92a8-37a4a312dc87", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90322025-04-30T06:13:25.423ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90332025-04-30T06:13:25.423ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
90342025-04-30T06:13:25.424ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90352025-04-30T06:13:25.424ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
90362025-04-30T06:13:25.430ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "caf4f7a5-078c-4e5c-b816-4157f8c98a76", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90372025-04-30T06:13:25.430ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
90382025-04-30T06:13:25.452ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d703367-c125-453e-ac4b-ec4efd2133f7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90392025-04-30T06:13:25.454ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
90402025-04-30T06:13:25.463ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eebf977e-2559-4601-a8d0-327f5a73cf67", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90412025-04-30T06:13:25.464ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
90422025-04-30T06:13:25.467ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c04cad5-afb0-43e1-8f01-fdef5555b02f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90432025-04-30T06:13:25.467ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
90442025-04-30T06:13:25.471ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a82b1aef-76c9-460e-90f4-bd712cb895b5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90452025-04-30T06:13:25.471ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
90462025-04-30T06:13:25.473ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f923d90-724c-45cd-8881-9b47187532ae", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
90472025-04-30T06:13:25.473ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
90482025-04-30T06:13:25.474ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f8b6906-6248-4aba-925c-b64c4fd21f86", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90492025-04-30T06:13:25.474ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
90502025-04-30T06:13:25.476ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cee377c0-8726-41a8-a780-1c6f45d0924c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90512025-04-30T06:13:25.476ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
90522025-04-30T06:13:25.478ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0bdf1b94-ca6d-4bc2-ba43-8ca7a5646b9e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:24 GMT"} })
90532025-04-30T06:13:25.479ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
90542025-04-30T06:13:25.479ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90552025-04-30T06:13:25.479ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90562025-04-30T06:13:25.479ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90572025-04-30T06:13:25.481ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90582025-04-30T06:13:25.498ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90592025-04-30T06:13:25.499ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
90602025-04-30T06:13:25.503ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
90612025-04-30T06:13:25.503ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
90622025-04-30T06:13:25.503ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90632025-04-30T06:13:25.503ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90642025-04-30T06:13:25.503ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90652025-04-30T06:13:25.504ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90662025-04-30T06:13:25.508ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90672025-04-30T06:13:25.550ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
90682025-04-30T06:13:25.550ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90692025-04-30T06:13:25.550ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90702025-04-30T06:13:25.550ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
90712025-04-30T06:13:25.551ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90722025-04-30T06:13:25.567ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
90732025-04-30T06:13:25.567ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90742025-04-30T06:13:25.567ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90752025-04-30T06:13:25.585ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90762025-04-30T06:13:25.594ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90772025-04-30T06:13:25.602ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
90782025-04-30T06:13:25.602ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90792025-04-30T06:13:25.602ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
90802025-04-30T06:13:25.602ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90812025-04-30T06:13:25.636ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
90822025-04-30T06:13:25.637ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
90832025-04-30T06:13:25.638ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:58511 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
90842025-04-30T06:13:25.639ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:58511 req_id = ba6ca138-3249-434c-b3fb-7653c426db51 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
90852025-04-30T06:13:25.640ZINFOomicron-dev (dropshot): request completed latency_us = 576 local_addr = [::1]:61881 method = GET remote_addr = [::1]:58511 req_id = ba6ca138-3249-434c-b3fb-7653c426db51 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
90862025-04-30T06:13:25.641ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
90872025-04-30T06:13:25.641ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
90882025-04-30T06:13:25.642ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba6ca138-3249-434c-b3fb-7653c426db51", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
90892025-04-30T06:13:25.643ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
90902025-04-30T06:13:25.644ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
90912025-04-30T06:13:25.645ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
90922025-04-30T06:13:25.645ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
90932025-04-30T06:13:25.648ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
90942025-04-30T06:13:25.648ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
90952025-04-30T06:13:25.649ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
90962025-04-30T06:13:25.649ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
90972025-04-30T06:13:25.650ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
90982025-04-30T06:13:25.675ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
90992025-04-30T06:13:25.698ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
91002025-04-30T06:13:25.704ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
91012025-04-30T06:13:25.704ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
91022025-04-30T06:13:25.705ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91032025-04-30T06:13:25.706ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
91042025-04-30T06:13:25.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91052025-04-30T06:13:25.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91062025-04-30T06:13:25.723ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91072025-04-30T06:13:25.724ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91082025-04-30T06:13:25.724ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91092025-04-30T06:13:25.724ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
91102025-04-30T06:13:25.750ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
91112025-04-30T06:13:25.750ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
91122025-04-30T06:13:25.750ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91132025-04-30T06:13:25.750ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
91142025-04-30T06:13:25.751ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91152025-04-30T06:13:25.764ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91162025-04-30T06:13:25.769ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
91172025-04-30T06:13:25.769ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91182025-04-30T06:13:25.769ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
91192025-04-30T06:13:25.769ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91202025-04-30T06:13:25.772ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
91212025-04-30T06:13:25.772ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
91222025-04-30T06:13:25.772ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 614.558277ms iteration = 30 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
91232025-04-30T06:13:25.772ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 31 reason = Signaled
91242025-04-30T06:13:25.772ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
91252025-04-30T06:13:25.772ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
91262025-04-30T06:13:25.772ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
91272025-04-30T06:13:25.772ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91282025-04-30T06:13:25.772ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91292025-04-30T06:13:25.772ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
91302025-04-30T06:13:25.773ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91312025-04-30T06:13:25.773ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
91322025-04-30T06:13:25.783ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91332025-04-30T06:13:25.791ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
91342025-04-30T06:13:25.791ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
91352025-04-30T06:13:25.791ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
91362025-04-30T06:13:25.792ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
91372025-04-30T06:13:25.803ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
91382025-04-30T06:13:25.810ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
91392025-04-30T06:13:25.811ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
91402025-04-30T06:13:25.812ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 0e cb 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:65479 req_id = a71d9fc3-2438-4b2f-870c-f58900ca2e34
91412025-04-30T06:13:25.812ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 3787,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65479 req_id = a71d9fc3-2438-4b2f-870c-f58900ca2e34
91422025-04-30T06:13:25.812ZDEBGomicron-dev (store): query key key = _mgs._tcp
91432025-04-30T06:13:25.813ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
91442025-04-30T06:13:25.813ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65479 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = a71d9fc3-2438-4b2f-870c-f58900ca2e34
91452025-04-30T06:13:25.813ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2470, tv_nsec: 612543725 } })
91462025-04-30T06:13:25.813ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
91472025-04-30T06:13:25.814ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 80 f4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64533 req_id = 21b54401-9cbe-402f-ac83-71394460b1a8
91482025-04-30T06:13:25.814ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33012,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64533 req_id = 21b54401-9cbe-402f-ac83-71394460b1a8
91492025-04-30T06:13:25.814ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
91502025-04-30T06:13:25.814ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64533 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 21b54401-9cbe-402f-ac83-71394460b1a8
91512025-04-30T06:13:25.874ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
91522025-04-30T06:13:25.874ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
91532025-04-30T06:13:25.875ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
91542025-04-30T06:13:25.875ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91552025-04-30T06:13:25.875ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
91562025-04-30T06:13:25.875ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 15 b5 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58191 req_id = 20631289-43a6-4240-a7ad-ab045df0ba5d
91572025-04-30T06:13:25.875ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5557,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58191 req_id = 20631289-43a6-4240-a7ad-ab045df0ba5d
91582025-04-30T06:13:25.875ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
91592025-04-30T06:13:25.875ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91602025-04-30T06:13:25.875ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5557, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58191 req_id = 20631289-43a6-4240-a7ad-ab045df0ba5d
91612025-04-30T06:13:25.876ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
91622025-04-30T06:13:25.876ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
91632025-04-30T06:13:25.876ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
91642025-04-30T06:13:25.879ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
91652025-04-30T06:13:25.880ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "249c155f-1e62-4130-afef-b454fcfbf8dd", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91662025-04-30T06:13:25.880ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
91672025-04-30T06:13:25.882ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "20277ce8-6426-4640-af82-28dc73c2f0af", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91682025-04-30T06:13:25.882ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
91692025-04-30T06:13:25.884ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef120c3f-5d71-49ea-b100-a9bbca8dc217", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91702025-04-30T06:13:25.884ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
91712025-04-30T06:13:25.885ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
91722025-04-30T06:13:25.885ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91732025-04-30T06:13:25.885ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
91742025-04-30T06:13:25.886ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91752025-04-30T06:13:25.886ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c671e87-ebf0-4d97-bc53-fd889592e517", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91762025-04-30T06:13:25.887ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
91772025-04-30T06:13:25.888ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "99c013dc-b3c5-4daa-a995-ebcb78396960", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91782025-04-30T06:13:25.889ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
91792025-04-30T06:13:25.890ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9a9116a-fcb2-4fe7-b0dd-193c849dee44", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91802025-04-30T06:13:25.890ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
91812025-04-30T06:13:25.892ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ded7ecb-11d6-49cf-942f-02e0c632d2df", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91822025-04-30T06:13:25.892ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
91832025-04-30T06:13:25.895ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
91842025-04-30T06:13:25.895ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
91852025-04-30T06:13:25.896ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
91862025-04-30T06:13:25.897ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9aa218b-4f69-40fc-a2c8-c8a8ce7f21c3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91872025-04-30T06:13:25.897ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
91882025-04-30T06:13:25.898ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc60e4b3-e088-41be-b7bd-5f4de8328802", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91892025-04-30T06:13:25.898ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
91902025-04-30T06:13:25.901ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbbbadf8-b6d9-4ee8-b4aa-d9430f549ee9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91912025-04-30T06:13:25.901ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
91922025-04-30T06:13:25.903ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "92ba1379-877d-479e-87e2-4fb21ff10f73", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91932025-04-30T06:13:25.903ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
91942025-04-30T06:13:25.904ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f26f662-eb7c-4dfd-92dd-6590fdd11ad8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91952025-04-30T06:13:25.904ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
91962025-04-30T06:13:25.905ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e97d86d9-dc28-4ddd-8ff2-3c358e07a1a2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91972025-04-30T06:13:25.905ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
91982025-04-30T06:13:25.910ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ece77f1-e6e6-4509-a846-9b3c47781556", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
91992025-04-30T06:13:25.910ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
92002025-04-30T06:13:25.912ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2125a95b-f1ee-4ea5-8392-a36f47508563", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92012025-04-30T06:13:25.913ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
92022025-04-30T06:13:25.915ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78d4a75c-1a03-426c-b64a-26ec56a60c51", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92032025-04-30T06:13:25.915ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
92042025-04-30T06:13:25.917ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "69d7df3c-9bd4-4eb3-b916-38d83c2acfe2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92052025-04-30T06:13:25.917ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
92062025-04-30T06:13:25.920ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9dfc7de2-099d-4fdc-8913-00e40e3055c2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92072025-04-30T06:13:25.920ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
92082025-04-30T06:13:25.922ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3afb885b-f001-4f77-bb9a-7da5a626db1f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92092025-04-30T06:13:25.922ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
92102025-04-30T06:13:25.923ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fb57b2e-5623-4cba-addc-f3884877eb92", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92112025-04-30T06:13:25.923ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
92122025-04-30T06:13:25.924ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ca8328b-b24f-4b90-9755-b13b55999057", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92132025-04-30T06:13:25.924ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
92142025-04-30T06:13:25.925ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f49c697-a551-4dd2-b105-19e192c2d5ac", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92152025-04-30T06:13:25.925ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
92162025-04-30T06:13:25.927ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "27e7481b-f25a-40eb-9f73-91a694856397", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92172025-04-30T06:13:25.927ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
92182025-04-30T06:13:25.932ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62edaeb3-0792-4699-b486-746368a00ded", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92192025-04-30T06:13:25.932ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
92202025-04-30T06:13:25.936ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "93162847-63df-4940-91e4-adfaa36a1f7e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92212025-04-30T06:13:25.936ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
92222025-04-30T06:13:25.939ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a87183e9-35a0-4401-8ecc-51c26c49e0cc", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92232025-04-30T06:13:25.939ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
92242025-04-30T06:13:25.941ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4826b75f-05cf-4fd1-8593-5e7df1dedd71", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92252025-04-30T06:13:25.942ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
92262025-04-30T06:13:25.944ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1175c1c3-4b52-48d5-80ce-b4167b39f663", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92272025-04-30T06:13:25.944ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
92282025-04-30T06:13:25.946ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa425b9a-b3ae-492a-9d7e-409a8200f2fa", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92292025-04-30T06:13:25.946ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
92302025-04-30T06:13:25.948ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3fdfe274-5457-422e-8009-a00be07e109c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92312025-04-30T06:13:25.949ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
92322025-04-30T06:13:25.951ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86378650-7d5b-4bb5-83d5-dbaf221df1fb", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92332025-04-30T06:13:25.951ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
92342025-04-30T06:13:25.951ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "87445ebc-f5c4-40d8-b1f5-36b847a52544", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92352025-04-30T06:13:25.951ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
92362025-04-30T06:13:25.953ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fdaf09f-7ced-4d14-93be-eb6e064e2a09", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92372025-04-30T06:13:25.953ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
92382025-04-30T06:13:25.954ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62f25b2a-5f57-4bf5-aaec-6e2678a78347", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92392025-04-30T06:13:25.954ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
92402025-04-30T06:13:25.955ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd76720d-6698-4166-af99-87c8b365efab", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92412025-04-30T06:13:25.955ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
92422025-04-30T06:13:25.957ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "303516da-1627-4189-b2e9-323f935644ce", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92432025-04-30T06:13:25.957ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
92442025-04-30T06:13:25.970ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "17c55a4e-4d64-474f-a839-bed3f740c15a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92452025-04-30T06:13:25.970ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
92462025-04-30T06:13:25.973ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "477a0ee0-0d40-4a58-b30a-ded8a816d809", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92472025-04-30T06:13:25.973ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
92482025-04-30T06:13:25.982ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a60b233-1b97-4173-988b-b66b247ccb70", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92492025-04-30T06:13:25.982ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
92502025-04-30T06:13:25.988ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7275bd41-22c9-44c6-a492-bd9867158a09", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92512025-04-30T06:13:25.989ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
92522025-04-30T06:13:25.993ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0654d03d-919c-4467-a85b-5e17aeebdb50", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92532025-04-30T06:13:25.994ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
92542025-04-30T06:13:25.997ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8769b87-0e09-4331-bfda-ee98a48d709f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92552025-04-30T06:13:26.009ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
92562025-04-30T06:13:26.023ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "620964f5-c122-415d-b66f-13d18be5e3bc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92572025-04-30T06:13:26.024ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
92582025-04-30T06:13:26.025ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc2696b3-3890-4d5c-97a9-c3ce5feb2119", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92592025-04-30T06:13:26.025ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
92602025-04-30T06:13:26.026ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a1a3e1e-066d-41d9-97c9-f2f1fb77029e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92612025-04-30T06:13:26.026ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
92622025-04-30T06:13:26.026ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92632025-04-30T06:13:26.026ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92642025-04-30T06:13:26.026ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92652025-04-30T06:13:26.027ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92662025-04-30T06:13:26.050ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92672025-04-30T06:13:26.059ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
92682025-04-30T06:13:26.059ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
92692025-04-30T06:13:26.059ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92702025-04-30T06:13:26.059ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92712025-04-30T06:13:26.059ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92722025-04-30T06:13:26.060ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92732025-04-30T06:13:26.062ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
92742025-04-30T06:13:26.071ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92752025-04-30T06:13:26.091ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
92762025-04-30T06:13:26.091ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
92772025-04-30T06:13:26.091ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
92782025-04-30T06:13:26.091ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
92792025-04-30T06:13:26.091ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
92802025-04-30T06:13:26.091ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
92812025-04-30T06:13:26.092ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
92822025-04-30T06:13:26.093ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
92832025-04-30T06:13:26.098ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
92842025-04-30T06:13:26.098ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
92852025-04-30T06:13:26.107ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
92862025-04-30T06:13:26.107ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
92872025-04-30T06:13:26.121ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
92882025-04-30T06:13:26.121ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
92892025-04-30T06:13:26.121ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
92902025-04-30T06:13:26.121ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
92912025-04-30T06:13:26.122ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
92922025-04-30T06:13:26.173ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
92932025-04-30T06:13:26.173ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
92942025-04-30T06:13:26.174ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:64015 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
92952025-04-30T06:13:26.175ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:64015 req_id = 40661ef8-75d8-4772-9a17-61ceb30d02ff sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
92962025-04-30T06:13:26.176ZINFOomicron-dev (dropshot): request completed latency_us = 1173 local_addr = [::1]:61881 method = GET remote_addr = [::1]:64015 req_id = 40661ef8-75d8-4772-9a17-61ceb30d02ff response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
92972025-04-30T06:13:26.177ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "40661ef8-75d8-4772-9a17-61ceb30d02ff", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
92982025-04-30T06:13:26.179ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
92992025-04-30T06:13:26.180ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
93002025-04-30T06:13:26.180ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
93012025-04-30T06:13:26.180ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
93022025-04-30T06:13:26.182ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
93032025-04-30T06:13:26.182ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
93042025-04-30T06:13:26.182ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
93052025-04-30T06:13:26.182ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
93062025-04-30T06:13:26.183ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
93072025-04-30T06:13:26.185ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
93082025-04-30T06:13:26.185ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93092025-04-30T06:13:26.185ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
93102025-04-30T06:13:26.186ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93112025-04-30T06:13:26.192ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
93122025-04-30T06:13:26.195ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93132025-04-30T06:13:26.197ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93142025-04-30T06:13:26.197ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
93152025-04-30T06:13:26.198ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93162025-04-30T06:13:26.204ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
93172025-04-30T06:13:26.205ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
93182025-04-30T06:13:26.205ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
93192025-04-30T06:13:26.205ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
93202025-04-30T06:13:26.242ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
93212025-04-30T06:13:26.279ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
93222025-04-30T06:13:26.279ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
93232025-04-30T06:13:26.279ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 506.913001ms iteration = 31 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
93242025-04-30T06:13:26.279ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 32 reason = Signaled
93252025-04-30T06:13:26.279ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
93262025-04-30T06:13:26.279ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
93272025-04-30T06:13:26.279ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
93282025-04-30T06:13:26.279ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
93292025-04-30T06:13:26.280ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
93302025-04-30T06:13:26.283ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93312025-04-30T06:13:26.288ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
93322025-04-30T06:13:26.288ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
93332025-04-30T06:13:26.289ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
93342025-04-30T06:13:26.289ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
93352025-04-30T06:13:26.293ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
93362025-04-30T06:13:26.293ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
93372025-04-30T06:13:26.293ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
93382025-04-30T06:13:26.293ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
93392025-04-30T06:13:26.293ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93402025-04-30T06:13:26.293ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
93412025-04-30T06:13:26.293ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 1e 92 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56288 req_id = 5267163d-caa2-4750-8fb8-eb3ac3040001
93422025-04-30T06:13:26.293ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7826,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56288 req_id = 5267163d-caa2-4750-8fb8-eb3ac3040001
93432025-04-30T06:13:26.294ZDEBGomicron-dev (store): query key key = _mgs._tcp
93442025-04-30T06:13:26.294ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
93452025-04-30T06:13:26.294ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93462025-04-30T06:13:26.294ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56288 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5267163d-caa2-4750-8fb8-eb3ac3040001
93472025-04-30T06:13:26.294ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2471, tv_nsec: 93611822 } })
93482025-04-30T06:13:26.294ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
93492025-04-30T06:13:26.295ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 70 21 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e p!..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65123 req_id = 46359fdc-6caa-4b0c-8097-7b09d01e0971
93502025-04-30T06:13:26.295ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28705,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65123 req_id = 46359fdc-6caa-4b0c-8097-7b09d01e0971
93512025-04-30T06:13:26.295ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
93522025-04-30T06:13:26.295ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65123 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 46359fdc-6caa-4b0c-8097-7b09d01e0971
93532025-04-30T06:13:26.297ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
93542025-04-30T06:13:26.303ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
93552025-04-30T06:13:26.303ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93562025-04-30T06:13:26.303ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
93572025-04-30T06:13:26.303ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93582025-04-30T06:13:26.315ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
93592025-04-30T06:13:26.315ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
93602025-04-30T06:13:26.315ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
93612025-04-30T06:13:26.349ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
93622025-04-30T06:13:26.358ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
93632025-04-30T06:13:26.359ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ac a8 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51929 req_id = 360a00af-38ef-439e-892f-46af7974497b
93642025-04-30T06:13:26.359ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44200,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51929 req_id = 360a00af-38ef-439e-892f-46af7974497b
93652025-04-30T06:13:26.359ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
93662025-04-30T06:13:26.359ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44200, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51929 req_id = 360a00af-38ef-439e-892f-46af7974497b
93672025-04-30T06:13:26.360ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
93682025-04-30T06:13:26.360ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
93692025-04-30T06:13:26.360ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
93702025-04-30T06:13:26.362ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "27bd6aba-086b-4750-9f30-f4f3fc0eb191", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93712025-04-30T06:13:26.362ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
93722025-04-30T06:13:26.364ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "10417171-62f6-48b8-ab26-860ba034f9b8", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93732025-04-30T06:13:26.364ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
93742025-04-30T06:13:26.366ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a601505-88b7-4b77-98b0-c2b4322fc93c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93752025-04-30T06:13:26.366ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
93762025-04-30T06:13:26.368ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fc24864-05db-4537-b378-4cb80df515b0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93772025-04-30T06:13:26.368ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
93782025-04-30T06:13:26.370ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fed374a5-ea5a-440f-99e8-3aaadda1570d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93792025-04-30T06:13:26.370ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
93802025-04-30T06:13:26.372ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80bb6942-289e-4c6a-8af8-14a26962eb2a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93812025-04-30T06:13:26.372ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
93822025-04-30T06:13:26.374ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eb47059-31cd-4c4c-9ffd-bd59757c0300", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93832025-04-30T06:13:26.374ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
93842025-04-30T06:13:26.378ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "113b3527-6925-43e0-8898-3208c05ad1fd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
93852025-04-30T06:13:26.379ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
93862025-04-30T06:13:26.380ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbc5f696-54ce-467b-a558-9eaae4757e7a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
93872025-04-30T06:13:26.380ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
93882025-04-30T06:13:26.382ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff5cb6a1-97c6-40f6-af4c-5004a1a00925", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93892025-04-30T06:13:26.382ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
93902025-04-30T06:13:26.383ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "abec9f67-3b3e-495e-88c8-1b4ff4109ec2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93912025-04-30T06:13:26.384ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
93922025-04-30T06:13:26.385ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33f4407c-942a-4a1e-851b-2aa0c271fe06", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93932025-04-30T06:13:26.385ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
93942025-04-30T06:13:26.386ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "96d0006f-132f-4f7f-bce4-88d07c722855", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93952025-04-30T06:13:26.387ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
93962025-04-30T06:13:26.389ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6cd3a40-ed2b-46f3-b7b0-8b344334a9c0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93972025-04-30T06:13:26.389ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
93982025-04-30T06:13:26.391ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a79cab19-e066-4e19-8aac-6c75fac3b2e2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
93992025-04-30T06:13:26.391ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
94002025-04-30T06:13:26.400ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "558c5edc-e50d-4d16-afae-4d70778caa8c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94012025-04-30T06:13:26.400ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
94022025-04-30T06:13:26.401ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
94032025-04-30T06:13:26.401ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94042025-04-30T06:13:26.401ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94052025-04-30T06:13:26.401ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
94062025-04-30T06:13:26.402ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94072025-04-30T06:13:26.402ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b4a6670-f8a0-4c8f-8a4a-da21ea5a6d23", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94082025-04-30T06:13:26.402ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
94092025-04-30T06:13:26.405ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e302e00-ad55-4529-9e29-484ca3caa7e6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94102025-04-30T06:13:26.405ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
94112025-04-30T06:13:26.406ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94122025-04-30T06:13:26.411ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
94132025-04-30T06:13:26.411ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94142025-04-30T06:13:26.411ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
94152025-04-30T06:13:26.411ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94162025-04-30T06:13:26.416ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94172025-04-30T06:13:26.416ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
94182025-04-30T06:13:26.417ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94192025-04-30T06:13:26.427ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e359342-4726-4422-bd4f-e35d4938c7ba", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94202025-04-30T06:13:26.428ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
94212025-04-30T06:13:26.432ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "27b1d331-a6da-47d0-b730-ce38fb009324", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94222025-04-30T06:13:26.432ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
94232025-04-30T06:13:26.432ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bb2837a-82b8-4a94-a898-2e5d1c4c76dd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94242025-04-30T06:13:26.432ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
94252025-04-30T06:13:26.432ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e76302ae-e6d9-497a-907f-cdeb5b275bb7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94262025-04-30T06:13:26.432ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
94272025-04-30T06:13:26.434ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "51fef2a4-d362-420b-99e7-945ae203c9ab", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94282025-04-30T06:13:26.434ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
94292025-04-30T06:13:26.436ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3a5ccbb-88a8-422f-9216-0ce77eb2c182", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94302025-04-30T06:13:26.436ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
94312025-04-30T06:13:26.440ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a656829-d7ba-4540-8c89-0bf4af9f9c44", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94322025-04-30T06:13:26.440ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
94332025-04-30T06:13:26.442ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "52b2a2aa-3843-4300-a233-fc2b78fb5040", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94342025-04-30T06:13:26.442ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
94352025-04-30T06:13:26.444ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d42d1de5-947f-4558-bf23-7849460eb7c8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94362025-04-30T06:13:26.444ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
94372025-04-30T06:13:26.446ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3fdaaf0-84a2-4b40-8428-31ba0aa30ad8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94382025-04-30T06:13:26.446ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
94392025-04-30T06:13:26.448ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "09da08ed-88d3-415c-8a79-aeb6a127e373", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94402025-04-30T06:13:26.448ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
94412025-04-30T06:13:26.450ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f00c1677-f241-4801-a7bc-ac59ccf95022", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94422025-04-30T06:13:26.451ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
94432025-04-30T06:13:26.452ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e6c075a-4957-4cc8-9cdc-0e9e342574b4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94442025-04-30T06:13:26.452ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
94452025-04-30T06:13:26.454ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dccb47f-386d-42c8-b29a-d45b86b8c493", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94462025-04-30T06:13:26.454ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
94472025-04-30T06:13:26.455ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "018669ff-de35-44cf-966a-0a8d738855e5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94482025-04-30T06:13:26.457ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
94492025-04-30T06:13:26.458ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
94502025-04-30T06:13:26.459ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b211b62b-242a-4b78-b48c-ead456ba6bdd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
94512025-04-30T06:13:26.459ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
94522025-04-30T06:13:26.461ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1daee9de-af8a-45d5-91b5-b55a54d35933", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94532025-04-30T06:13:26.461ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
94542025-04-30T06:13:26.465ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d337ab3-b8ad-4609-a698-bd8b141a979f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94552025-04-30T06:13:26.466ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
94562025-04-30T06:13:26.472ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f35b1fa6-ad29-4adf-8519-20a313111b61", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94572025-04-30T06:13:26.484ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
94582025-04-30T06:13:26.509ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
94592025-04-30T06:13:26.509ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94602025-04-30T06:13:26.509ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94612025-04-30T06:13:26.509ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
94622025-04-30T06:13:26.510ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "532972de-3961-4c0a-bce3-07f46891c91b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94632025-04-30T06:13:26.510ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94642025-04-30T06:13:26.510ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
94652025-04-30T06:13:26.513ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94662025-04-30T06:13:26.518ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
94672025-04-30T06:13:26.518ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94682025-04-30T06:13:26.518ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
94692025-04-30T06:13:26.518ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94702025-04-30T06:13:26.536ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
94712025-04-30T06:13:26.536ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
94722025-04-30T06:13:26.537ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "503b9424-8908-4cd4-a1b1-8c8eeaea62b6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94732025-04-30T06:13:26.538ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
94742025-04-30T06:13:26.538ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
94752025-04-30T06:13:26.543ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8d3fea1-74ef-4017-a657-7457c547de2c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94762025-04-30T06:13:26.543ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
94772025-04-30T06:13:26.552ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "da1b0de2-440d-49f1-89da-e4de6baa0ef3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94782025-04-30T06:13:26.553ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
94792025-04-30T06:13:26.554ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cefa9e7a-12c7-4e36-921b-dbc0c7107f7c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94802025-04-30T06:13:26.554ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
94812025-04-30T06:13:26.565ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d992623-0425-4efc-9ff8-a784a2b9a2dd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94822025-04-30T06:13:26.566ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
94832025-04-30T06:13:26.568ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "832b096b-06b4-45a9-97ac-319007fed64a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94842025-04-30T06:13:26.568ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
94852025-04-30T06:13:26.569ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0060e16d-2a07-45b0-8d30-6607104c9d68", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
94862025-04-30T06:13:26.569ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
94872025-04-30T06:13:26.570ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94882025-04-30T06:13:26.570ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94892025-04-30T06:13:26.570ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94902025-04-30T06:13:26.570ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94912025-04-30T06:13:26.574ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
94922025-04-30T06:13:26.575ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
94932025-04-30T06:13:26.580ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
94942025-04-30T06:13:26.580ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
94952025-04-30T06:13:26.581ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
94962025-04-30T06:13:26.581ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
94972025-04-30T06:13:26.581ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
94982025-04-30T06:13:26.581ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
94992025-04-30T06:13:26.585ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95002025-04-30T06:13:26.594ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
95012025-04-30T06:13:26.598ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
95022025-04-30T06:13:26.599ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
95032025-04-30T06:13:26.599ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
95042025-04-30T06:13:26.626ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
95052025-04-30T06:13:26.626ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
95062025-04-30T06:13:26.626ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95072025-04-30T06:13:26.626ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
95082025-04-30T06:13:26.627ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95092025-04-30T06:13:26.639ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
95102025-04-30T06:13:26.639ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
95112025-04-30T06:13:26.639ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:62692 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
95122025-04-30T06:13:26.640ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:62692 req_id = 50bde872-cc55-40e1-9055-6e120b53a038 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
95132025-04-30T06:13:26.641ZINFOomicron-dev (dropshot): request completed latency_us = 1225 local_addr = [::1]:61881 method = GET remote_addr = [::1]:62692 req_id = 50bde872-cc55-40e1-9055-6e120b53a038 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
95142025-04-30T06:13:26.641ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "50bde872-cc55-40e1-9055-6e120b53a038", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
95152025-04-30T06:13:26.644ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
95162025-04-30T06:13:26.644ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
95172025-04-30T06:13:26.644ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
95182025-04-30T06:13:26.645ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95192025-04-30T06:13:26.646ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
95202025-04-30T06:13:26.646ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
95212025-04-30T06:13:26.646ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
95222025-04-30T06:13:26.646ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
95232025-04-30T06:13:26.647ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
95242025-04-30T06:13:26.659ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
95252025-04-30T06:13:26.659ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95262025-04-30T06:13:26.659ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
95272025-04-30T06:13:26.660ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95282025-04-30T06:13:26.664ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95292025-04-30T06:13:26.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
95302025-04-30T06:13:26.679ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95312025-04-30T06:13:26.680ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = b5ccff18-a650-4c50-9f3e-9fb9c8310c20 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95322025-04-30T06:13:26.680ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = b5ccff18-a650-4c50-9f3e-9fb9c8310c20 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95332025-04-30T06:13:26.680ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = b5ccff18-a650-4c50-9f3e-9fb9c8310c20 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95342025-04-30T06:13:26.681ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = b5ccff18-a650-4c50-9f3e-9fb9c8310c20 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95352025-04-30T06:13:26.682ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
95362025-04-30T06:13:26.682ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
95372025-04-30T06:13:26.683ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
95382025-04-30T06:13:26.687ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
95392025-04-30T06:13:26.710ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95402025-04-30T06:13:26.710ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
95412025-04-30T06:13:26.711ZINFOomicron-dev (dropshot_internal): request completed latency_us = 30919 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = b5ccff18-a650-4c50-9f3e-9fb9c8310c20 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
95422025-04-30T06:13:26.714ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5ccff18-a650-4c50-9f3e-9fb9c8310c20", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
95432025-04-30T06:13:26.714ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
95442025-04-30T06:13:26.714ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95452025-04-30T06:13:26.715ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 493750cc-7ccf-468d-9977-4fffc01ba278 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
95462025-04-30T06:13:26.716ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 493750cc-7ccf-468d-9977-4fffc01ba278 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
95472025-04-30T06:13:26.716ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 493750cc-7ccf-468d-9977-4fffc01ba278 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
95482025-04-30T06:13:26.716ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 493750cc-7ccf-468d-9977-4fffc01ba278 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
95492025-04-30T06:13:26.721ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95502025-04-30T06:13:26.722ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95512025-04-30T06:13:26.727ZINFOomicron-dev (dropshot_internal): request completed latency_us = 11776 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 493750cc-7ccf-468d-9977-4fffc01ba278 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
95522025-04-30T06:13:26.728ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95532025-04-30T06:13:26.730ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "493750cc-7ccf-468d-9977-4fffc01ba278", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:25 GMT"} })
95542025-04-30T06:13:26.730ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95552025-04-30T06:13:26.730ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95562025-04-30T06:13:26.730ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
95572025-04-30T06:13:26.731ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
95582025-04-30T06:13:26.731ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95592025-04-30T06:13:26.731ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
95602025-04-30T06:13:26.799ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
95612025-04-30T06:13:26.801ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
95622025-04-30T06:13:26.801ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 522.184659ms iteration = 32 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
95632025-04-30T06:13:26.801ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 33 reason = Signaled
95642025-04-30T06:13:26.801ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
95652025-04-30T06:13:26.801ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
95662025-04-30T06:13:26.801ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
95672025-04-30T06:13:26.802ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
95682025-04-30T06:13:26.802ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
95692025-04-30T06:13:26.806ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95702025-04-30T06:13:26.809ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
95712025-04-30T06:13:26.810ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
95722025-04-30T06:13:26.810ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
95732025-04-30T06:13:26.810ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
95742025-04-30T06:13:26.811ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
95752025-04-30T06:13:26.819ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
95762025-04-30T06:13:26.819ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
95772025-04-30T06:13:26.820ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 3a ad 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 :............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:58325 req_id = d07f8855-fbdc-4278-9aad-248f21476301
95782025-04-30T06:13:26.820ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15021,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58325 req_id = d07f8855-fbdc-4278-9aad-248f21476301
95792025-04-30T06:13:26.820ZDEBGomicron-dev (store): query key key = _mgs._tcp
95802025-04-30T06:13:26.820ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
95812025-04-30T06:13:26.821ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58325 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d07f8855-fbdc-4278-9aad-248f21476301
95822025-04-30T06:13:26.821ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2471, tv_nsec: 620527376 } })
95832025-04-30T06:13:26.821ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
95842025-04-30T06:13:26.822ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 62 7c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e b|..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57731 req_id = 6dad3b55-dbaa-45ab-a9b8-ca7d9fc927cb
95852025-04-30T06:13:26.822ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 25212,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57731 req_id = 6dad3b55-dbaa-45ab-a9b8-ca7d9fc927cb
95862025-04-30T06:13:26.824ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
95872025-04-30T06:13:26.824ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57731 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 6dad3b55-dbaa-45ab-a9b8-ca7d9fc927cb
95882025-04-30T06:13:26.867ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
95892025-04-30T06:13:26.867ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
95902025-04-30T06:13:26.867ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95912025-04-30T06:13:26.868ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
95922025-04-30T06:13:26.868ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95932025-04-30T06:13:26.873ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
95942025-04-30T06:13:26.880ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
95952025-04-30T06:13:26.880ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95962025-04-30T06:13:26.880ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
95972025-04-30T06:13:26.881ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
95982025-04-30T06:13:26.884ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
95992025-04-30T06:13:26.884ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
96002025-04-30T06:13:26.885ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
96012025-04-30T06:13:26.894ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
96022025-04-30T06:13:26.895ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 5b 96 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c [............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:57790 req_id = 67accbdf-7b90-4601-8719-8084de694577
96032025-04-30T06:13:26.895ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23446,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57790 req_id = 67accbdf-7b90-4601-8719-8084de694577
96042025-04-30T06:13:26.895ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
96052025-04-30T06:13:26.895ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 23446, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:57790 req_id = 67accbdf-7b90-4601-8719-8084de694577
96062025-04-30T06:13:26.899ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
96072025-04-30T06:13:26.899ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
96082025-04-30T06:13:26.899ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
96092025-04-30T06:13:26.901ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "16fbc579-c30e-4c12-a6da-253aa346e3f0", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96102025-04-30T06:13:26.902ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
96112025-04-30T06:13:26.904ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35bb6737-bbb0-4a03-b377-f4f4441318a2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96122025-04-30T06:13:26.904ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
96132025-04-30T06:13:26.906ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "acea7378-a99d-4967-ad88-01c89a5c3922", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96142025-04-30T06:13:26.906ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
96152025-04-30T06:13:26.909ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "abc3deed-52fb-4832-8870-5190a9d5dc40", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96162025-04-30T06:13:26.909ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
96172025-04-30T06:13:26.911ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ded62f34-f39a-4fd6-a0dc-007fbfb3914a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96182025-04-30T06:13:26.911ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
96192025-04-30T06:13:26.913ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "53240a47-6110-43a0-b84a-d1d40e7ce2d7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96202025-04-30T06:13:26.913ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
96212025-04-30T06:13:26.915ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe91604c-28b3-4145-8e01-febfe94df6ea", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96222025-04-30T06:13:26.915ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
96232025-04-30T06:13:26.917ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9669730-c7f4-458c-bc49-1ac5ff019c10", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96242025-04-30T06:13:26.917ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
96252025-04-30T06:13:26.918ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca5b7d96-10b2-4fa3-8c49-bb18d9f17ac2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96262025-04-30T06:13:26.918ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
96272025-04-30T06:13:26.919ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "50f35adb-f790-400e-9eef-9b3bfba5040c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96282025-04-30T06:13:26.920ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
96292025-04-30T06:13:26.921ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "607b0356-8883-493d-9676-1800a0281c42", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96302025-04-30T06:13:26.921ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
96312025-04-30T06:13:26.926ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b873fca-70a8-4f10-9176-3602ffd297cd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96322025-04-30T06:13:26.926ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
96332025-04-30T06:13:26.929ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b44fa21-21af-4c70-b70f-ce91fb50e961", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96342025-04-30T06:13:26.929ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
96352025-04-30T06:13:26.938ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d956041e-7c2c-42bb-909c-99504b3b68f0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96362025-04-30T06:13:26.938ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
96372025-04-30T06:13:26.939ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
96382025-04-30T06:13:26.945ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e64af89a-41e6-40ac-8eb1-901727d0f363", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96392025-04-30T06:13:26.945ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
96402025-04-30T06:13:26.949ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5caa8882-7fd0-45f1-a3ec-356b52be4bd2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96412025-04-30T06:13:26.949ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
96422025-04-30T06:13:26.953ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3edca8c4-4b73-4c86-a5a1-2ad2f2f61704", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96432025-04-30T06:13:26.954ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
96442025-04-30T06:13:26.961ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "61b8d97a-5040-4fcd-90e7-8d25ee132599", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96452025-04-30T06:13:26.964ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
96462025-04-30T06:13:26.969ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dee47437-9711-4005-8c06-0351029545b8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96472025-04-30T06:13:26.969ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
96482025-04-30T06:13:26.970ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e908741-6518-4010-a74e-b3023961fb60", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96492025-04-30T06:13:26.971ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
96502025-04-30T06:13:26.973ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f39839e8-567b-43af-94b0-7b256c6e2227", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96512025-04-30T06:13:26.973ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
96522025-04-30T06:13:26.974ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6ed22be-5dec-471d-9f1a-c8bfa8917f2e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96532025-04-30T06:13:26.974ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
96542025-04-30T06:13:26.975ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b8c2f2d-82a7-410a-a1ca-f8a033f5d1de", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96552025-04-30T06:13:26.975ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
96562025-04-30T06:13:26.977ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "93f8cc0b-0eca-4d69-b8b7-509be4cbf9e0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96572025-04-30T06:13:26.977ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
96582025-04-30T06:13:26.981ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "966d6ab8-48b2-4f68-9ae0-f9fdc22f2ef7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96592025-04-30T06:13:26.981ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
96602025-04-30T06:13:26.983ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "541725ff-8fe7-497a-8961-8f8019d5a86c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96612025-04-30T06:13:26.983ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
96622025-04-30T06:13:26.985ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "68a49641-8efb-47f5-b954-61168f6c63b6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96632025-04-30T06:13:26.985ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
96642025-04-30T06:13:26.987ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccb1de41-e3d4-451e-9e77-84395c00c5c7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96652025-04-30T06:13:26.987ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
96662025-04-30T06:13:26.989ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ad52301-4484-4e36-aa3f-fa9f2da6e6fc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96672025-04-30T06:13:26.990ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
96682025-04-30T06:13:26.992ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd23d52d-ac1c-45d9-9976-96e3af3af33e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96692025-04-30T06:13:26.992ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
96702025-04-30T06:13:26.993ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
96712025-04-30T06:13:26.993ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "505826fc-dc32-4c4d-b907-cb8a6216dd24", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96722025-04-30T06:13:26.995ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
96732025-04-30T06:13:26.995ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
96742025-04-30T06:13:26.995ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
96752025-04-30T06:13:26.995ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
96762025-04-30T06:13:26.995ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
96772025-04-30T06:13:26.996ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f2401f1-2b48-454c-91d3-043041404efc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96782025-04-30T06:13:26.996ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
96792025-04-30T06:13:26.997ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "625b257f-6594-45ef-b929-d457f0ec8ddf", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96802025-04-30T06:13:26.997ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
96812025-04-30T06:13:27.002ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "677a8506-5a07-408c-ae52-1729df137c9a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96822025-04-30T06:13:27.002ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
96832025-04-30T06:13:27.003ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "70cc4ff0-8b22-40f1-8c9a-a630b1ef9705", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96842025-04-30T06:13:27.003ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
96852025-04-30T06:13:27.013ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "61b9bc1c-bcbf-4fe6-88ce-ed3458675fc7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96862025-04-30T06:13:27.013ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
96872025-04-30T06:13:27.016ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "07a3a427-951f-4384-a468-1136bcd164bb", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96882025-04-30T06:13:27.016ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
96892025-04-30T06:13:27.025ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "24a27612-52a0-4a0e-81d9-43cc9b2abbf9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96902025-04-30T06:13:27.025ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
96912025-04-30T06:13:27.025ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
96922025-04-30T06:13:27.028ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "165696a4-6622-4ae5-9b5d-3932a1f5041f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96932025-04-30T06:13:27.028ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
96942025-04-30T06:13:27.030ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a472839e-a983-4abd-8317-2ca1afcc5621", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96952025-04-30T06:13:27.030ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
96962025-04-30T06:13:27.032ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "43523505-57c3-49f9-8805-42e1dea66dd0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96972025-04-30T06:13:27.032ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
96982025-04-30T06:13:27.034ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "79f8de53-2d7e-4f62-a955-9bb2489ad88d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
96992025-04-30T06:13:27.034ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
97002025-04-30T06:13:27.034ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
97012025-04-30T06:13:27.036ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8b0d0ce-49d3-4b46-9340-c0a3200e86f9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
97022025-04-30T06:13:27.036ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
97032025-04-30T06:13:27.037ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cfd18c5-1376-4cda-9960-bf010e3a3961", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
97042025-04-30T06:13:27.037ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
97052025-04-30T06:13:27.038ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aba46b12-8322-4b61-a6ed-dc31ec3a3342", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
97062025-04-30T06:13:27.039ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
97072025-04-30T06:13:27.039ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97082025-04-30T06:13:27.039ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97092025-04-30T06:13:27.039ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97102025-04-30T06:13:27.040ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97112025-04-30T06:13:27.040ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
97122025-04-30T06:13:27.041ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97132025-04-30T06:13:27.041ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97142025-04-30T06:13:27.047ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97152025-04-30T06:13:27.050ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97162025-04-30T06:13:27.050ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
97172025-04-30T06:13:27.051ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97182025-04-30T06:13:27.052ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
97192025-04-30T06:13:27.052ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
97202025-04-30T06:13:27.052ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97212025-04-30T06:13:27.052ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97222025-04-30T06:13:27.052ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97232025-04-30T06:13:27.053ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97242025-04-30T06:13:27.058ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97252025-04-30T06:13:27.061ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
97262025-04-30T06:13:27.061ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97272025-04-30T06:13:27.061ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97282025-04-30T06:13:27.062ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97292025-04-30T06:13:27.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
97302025-04-30T06:13:27.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
97312025-04-30T06:13:27.091ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
97322025-04-30T06:13:27.091ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
97332025-04-30T06:13:27.095ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
97342025-04-30T06:13:27.095ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
97352025-04-30T06:13:27.103ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
97362025-04-30T06:13:27.103ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
97372025-04-30T06:13:27.136ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
97382025-04-30T06:13:27.141ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
97392025-04-30T06:13:27.141ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
97402025-04-30T06:13:27.142ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:40444 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
97412025-04-30T06:13:27.142ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:40444 req_id = 7cb77296-bfb5-4cb3-b668-9bf69ce2b646 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
97422025-04-30T06:13:27.143ZINFOomicron-dev (dropshot): request completed latency_us = 853 local_addr = [::1]:61881 method = GET remote_addr = [::1]:40444 req_id = 7cb77296-bfb5-4cb3-b668-9bf69ce2b646 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
97432025-04-30T06:13:27.144ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cb77296-bfb5-4cb3-b668-9bf69ce2b646", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
97442025-04-30T06:13:27.149ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
97452025-04-30T06:13:27.149ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
97462025-04-30T06:13:27.149ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
97472025-04-30T06:13:27.153ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
97482025-04-30T06:13:27.153ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97492025-04-30T06:13:27.153ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97502025-04-30T06:13:27.153ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97512025-04-30T06:13:27.154ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97522025-04-30T06:13:27.163ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97532025-04-30T06:13:27.177ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
97542025-04-30T06:13:27.178ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97552025-04-30T06:13:27.178ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97562025-04-30T06:13:27.180ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97572025-04-30T06:13:27.187ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
97582025-04-30T06:13:27.187ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97592025-04-30T06:13:27.187ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97602025-04-30T06:13:27.187ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
97612025-04-30T06:13:27.188ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97622025-04-30T06:13:27.192ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
97632025-04-30T06:13:27.194ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97642025-04-30T06:13:27.202ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
97652025-04-30T06:13:27.202ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97662025-04-30T06:13:27.202ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
97672025-04-30T06:13:27.202ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97682025-04-30T06:13:27.229ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
97692025-04-30T06:13:27.230ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
97702025-04-30T06:13:27.230ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 428.447855ms iteration = 33 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
97712025-04-30T06:13:27.230ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 34 reason = Signaled
97722025-04-30T06:13:27.230ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
97732025-04-30T06:13:27.230ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
97742025-04-30T06:13:27.230ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97752025-04-30T06:13:27.230ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97762025-04-30T06:13:27.231ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97772025-04-30T06:13:27.233ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
97782025-04-30T06:13:27.236ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
97792025-04-30T06:13:27.237ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
97802025-04-30T06:13:27.237ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
97812025-04-30T06:13:27.239ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
97822025-04-30T06:13:27.239ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
97832025-04-30T06:13:27.239ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
97842025-04-30T06:13:27.239ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
97852025-04-30T06:13:27.244ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
97862025-04-30T06:13:27.245ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
97872025-04-30T06:13:27.245ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 6b d0 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 k............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:49684 req_id = 39834d5f-6838-4219-b966-ab087f9708a1
97882025-04-30T06:13:27.245ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27600,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49684 req_id = 39834d5f-6838-4219-b966-ab087f9708a1
97892025-04-30T06:13:27.246ZDEBGomicron-dev (store): query key key = _mgs._tcp
97902025-04-30T06:13:27.246ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
97912025-04-30T06:13:27.246ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:49684 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 39834d5f-6838-4219-b966-ab087f9708a1
97922025-04-30T06:13:27.246ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2472, tv_nsec: 45662360 } })
97932025-04-30T06:13:27.246ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
97942025-04-30T06:13:27.247ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f0 d3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52902 req_id = 50e11c3a-c199-4388-aaca-d3852e796c22
97952025-04-30T06:13:27.247ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61651,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52902 req_id = 50e11c3a-c199-4388-aaca-d3852e796c22
97962025-04-30T06:13:27.247ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
97972025-04-30T06:13:27.247ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52902 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 50e11c3a-c199-4388-aaca-d3852e796c22
97982025-04-30T06:13:27.280ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
97992025-04-30T06:13:27.311ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
98002025-04-30T06:13:27.314ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 43 61 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c Ca..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51025 req_id = ae963d35-efb6-4144-893a-4dca99f7f490
98012025-04-30T06:13:27.314ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17249,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51025 req_id = ae963d35-efb6-4144-893a-4dca99f7f490
98022025-04-30T06:13:27.314ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
98032025-04-30T06:13:27.314ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17249, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51025 req_id = ae963d35-efb6-4144-893a-4dca99f7f490
98042025-04-30T06:13:27.320ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
98052025-04-30T06:13:27.321ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
98062025-04-30T06:13:27.321ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
98072025-04-30T06:13:27.328ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "056d2caf-5206-494b-af50-dc5875ce581a", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98082025-04-30T06:13:27.329ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
98092025-04-30T06:13:27.331ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
98102025-04-30T06:13:27.332ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
98112025-04-30T06:13:27.332ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98122025-04-30T06:13:27.332ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
98132025-04-30T06:13:27.334ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "704dd50c-1ae1-4d6d-ba7a-dcfc8c9f00b4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98142025-04-30T06:13:27.335ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
98152025-04-30T06:13:27.335ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98162025-04-30T06:13:27.344ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98172025-04-30T06:13:27.346ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fec776d-07f5-4aec-bfb3-eb2975c62687", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98182025-04-30T06:13:27.357ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
98192025-04-30T06:13:27.357ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98202025-04-30T06:13:27.357ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
98212025-04-30T06:13:27.359ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98222025-04-30T06:13:27.359ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
98232025-04-30T06:13:27.362ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98242025-04-30T06:13:27.362ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
98252025-04-30T06:13:27.363ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98262025-04-30T06:13:27.375ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d3076b9-625f-4822-be90-b88b2d88a508", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98272025-04-30T06:13:27.375ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
98282025-04-30T06:13:27.384ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "49abbf91-b3c4-479f-8ecc-16551311c3a1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98292025-04-30T06:13:27.388ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
98302025-04-30T06:13:27.389ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0109f1f1-3a95-4641-aee2-961614765373", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98312025-04-30T06:13:27.389ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
98322025-04-30T06:13:27.392ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e84c20f-0306-4ff2-b337-9dcfd4359ce3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98332025-04-30T06:13:27.392ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
98342025-04-30T06:13:27.394ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c720577-b337-4c89-bb5c-ac1e8b53ae3a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98352025-04-30T06:13:27.395ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
98362025-04-30T06:13:27.397ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "205b560e-7c6c-44c9-95af-bd30ae6302dc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98372025-04-30T06:13:27.399ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
98382025-04-30T06:13:27.400ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33df00b5-b854-467e-99ad-5a447b432914", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98392025-04-30T06:13:27.400ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
98402025-04-30T06:13:27.404ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e661c35-c09a-4392-aa60-3fe5073c6986", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98412025-04-30T06:13:27.404ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
98422025-04-30T06:13:27.406ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "817e9cdc-1ea9-4fc6-b7d0-08c208148bae", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98432025-04-30T06:13:27.407ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
98442025-04-30T06:13:27.416ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cf1e18f5-8049-426b-baeb-835b3de96391", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98452025-04-30T06:13:27.417ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
98462025-04-30T06:13:27.427ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
98472025-04-30T06:13:27.428ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a9f6d08-50c7-41b9-8bba-0a323e7395e4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98482025-04-30T06:13:27.428ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
98492025-04-30T06:13:27.431ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4ba0c37-ca10-4ef9-b851-2bee664b8dc5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98502025-04-30T06:13:27.431ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
98512025-04-30T06:13:27.433ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d248a2d3-1c33-4515-b307-1fc713ee2cb4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98522025-04-30T06:13:27.433ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
98532025-04-30T06:13:27.435ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0e78aa2-1202-4d04-928a-4e6d8bf223bc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98542025-04-30T06:13:27.435ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
98552025-04-30T06:13:27.437ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e5687da-892d-4ae6-9e4c-485c772ffef7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98562025-04-30T06:13:27.437ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
98572025-04-30T06:13:27.439ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b3edfed-89b3-49dc-9b13-fe06b7f95b2f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98582025-04-30T06:13:27.439ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
98592025-04-30T06:13:27.440ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2e0b0cc-aa44-48b2-9438-602e7eabf94d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98602025-04-30T06:13:27.441ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
98612025-04-30T06:13:27.442ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b797765-2919-4b21-8681-9cdc4381c7c4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98622025-04-30T06:13:27.442ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
98632025-04-30T06:13:27.445ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c0952a5-9b6c-44a8-82cb-1d76b2896e88", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98642025-04-30T06:13:27.445ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
98652025-04-30T06:13:27.446ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f2b13d7-b732-4fa3-9825-80e9dc551e57", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98662025-04-30T06:13:27.447ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
98672025-04-30T06:13:27.449ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "333c3a4f-49d6-4959-b921-9ff873312dd7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98682025-04-30T06:13:27.449ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
98692025-04-30T06:13:27.452ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f1382a5-22e7-4c53-99e5-c641faa090a6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98702025-04-30T06:13:27.453ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
98712025-04-30T06:13:27.457ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb3bc50c-3c93-4340-9f55-f41751c72835", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98722025-04-30T06:13:27.457ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
98732025-04-30T06:13:27.461ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "68fe1405-126e-4144-b407-7ac4b72e7a45", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98742025-04-30T06:13:27.461ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
98752025-04-30T06:13:27.465ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "293ba004-552e-43ce-9ae7-35a6f283bef6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98762025-04-30T06:13:27.465ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
98772025-04-30T06:13:27.468ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ebb230f-4a43-4ee9-8832-66e7d31ef707", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98782025-04-30T06:13:27.468ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
98792025-04-30T06:13:27.470ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "70930737-718e-4867-9662-99a9bb03e2dd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98802025-04-30T06:13:27.470ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
98812025-04-30T06:13:27.471ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "348a3fa8-0b4d-4080-ad42-310c90cbd3c4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98822025-04-30T06:13:27.472ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
98832025-04-30T06:13:27.474ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fe7d9f4-6654-49d7-879b-587c0285bf82", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98842025-04-30T06:13:27.475ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
98852025-04-30T06:13:27.478ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
98862025-04-30T06:13:27.478ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
98872025-04-30T06:13:27.478ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
98882025-04-30T06:13:27.478ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
98892025-04-30T06:13:27.480ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
98902025-04-30T06:13:27.480ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ada54f4-a786-4d96-bab2-d903da481809", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98912025-04-30T06:13:27.480ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
98922025-04-30T06:13:27.482ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "83e53455-ecc4-4d22-8647-b40ecca60889", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98932025-04-30T06:13:27.482ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
98942025-04-30T06:13:27.483ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
98952025-04-30T06:13:27.483ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "91683bce-7a31-4233-8352-8b1e41f0d1db", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98962025-04-30T06:13:27.483ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
98972025-04-30T06:13:27.485ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03c3cd35-1ad7-4c48-b450-6fead78df70b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
98982025-04-30T06:13:27.486ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
98992025-04-30T06:13:27.488ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bbad22f-cb15-4a18-a656-fbb75177b203", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99002025-04-30T06:13:27.488ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
99012025-04-30T06:13:27.490ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f46acc29-26e0-42c2-8be3-27cb1a7ae1ac", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99022025-04-30T06:13:27.490ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
99032025-04-30T06:13:27.491ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
99042025-04-30T06:13:27.491ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99052025-04-30T06:13:27.491ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
99062025-04-30T06:13:27.492ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99072025-04-30T06:13:27.492ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d0809f6-a2cd-4071-9bb2-e8508cf01e4a", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99082025-04-30T06:13:27.492ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
99092025-04-30T06:13:27.494ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "341a6d78-8d63-4460-be95-9ecc4918923a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99102025-04-30T06:13:27.494ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
99112025-04-30T06:13:27.496ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c95d8095-3ece-4a4c-95d8-d5fdf7d42daf", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99122025-04-30T06:13:27.496ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
99132025-04-30T06:13:27.497ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8aa7c66e-caea-4ca8-adfb-5f84ef0bb32b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99142025-04-30T06:13:27.497ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
99152025-04-30T06:13:27.498ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a7c881f0-f7a4-4df4-8012-e0660ee4f70b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99162025-04-30T06:13:27.498ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
99172025-04-30T06:13:27.499ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "23b3a0aa-d840-4d50-9134-27f9033918fa", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99182025-04-30T06:13:27.499ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
99192025-04-30T06:13:27.500ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "088e755d-41a4-4f76-808a-ee1db66a6737", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99202025-04-30T06:13:27.501ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
99212025-04-30T06:13:27.501ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99222025-04-30T06:13:27.501ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99232025-04-30T06:13:27.501ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99242025-04-30T06:13:27.501ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99252025-04-30T06:13:27.502ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99262025-04-30T06:13:27.502ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
99272025-04-30T06:13:27.503ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99282025-04-30T06:13:27.505ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99292025-04-30T06:13:27.520ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
99302025-04-30T06:13:27.522ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
99312025-04-30T06:13:27.522ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99322025-04-30T06:13:27.522ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99332025-04-30T06:13:27.522ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99342025-04-30T06:13:27.528ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99352025-04-30T06:13:27.533ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99362025-04-30T06:13:27.538ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
99372025-04-30T06:13:27.539ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99382025-04-30T06:13:27.539ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99392025-04-30T06:13:27.539ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99402025-04-30T06:13:27.549ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
99412025-04-30T06:13:27.600ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
99422025-04-30T06:13:27.600ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99432025-04-30T06:13:27.600ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99442025-04-30T06:13:27.600ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
99452025-04-30T06:13:27.601ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99462025-04-30T06:13:27.660ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
99472025-04-30T06:13:27.661ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
99482025-04-30T06:13:27.661ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:46737 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
99492025-04-30T06:13:27.662ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:46737 req_id = bf5e40b5-0bae-4aaa-811e-12cd54baf7ee sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
99502025-04-30T06:13:27.662ZINFOomicron-dev (dropshot): request completed latency_us = 573 local_addr = [::1]:61881 method = GET remote_addr = [::1]:46737 req_id = bf5e40b5-0bae-4aaa-811e-12cd54baf7ee response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
99512025-04-30T06:13:27.663ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf5e40b5-0bae-4aaa-811e-12cd54baf7ee", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:26 GMT"} })
99522025-04-30T06:13:27.664ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99532025-04-30T06:13:27.665ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
99542025-04-30T06:13:27.666ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
99552025-04-30T06:13:27.666ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
99562025-04-30T06:13:27.668ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
99572025-04-30T06:13:27.668ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99582025-04-30T06:13:27.668ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99592025-04-30T06:13:27.668ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99602025-04-30T06:13:27.669ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99612025-04-30T06:13:27.670ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
99622025-04-30T06:13:27.670ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99632025-04-30T06:13:27.670ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
99642025-04-30T06:13:27.671ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99652025-04-30T06:13:27.672ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99662025-04-30T06:13:27.673ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
99672025-04-30T06:13:27.673ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
99682025-04-30T06:13:27.673ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
99692025-04-30T06:13:27.676ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
99702025-04-30T06:13:27.676ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99712025-04-30T06:13:27.676ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99722025-04-30T06:13:27.676ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99732025-04-30T06:13:27.711ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
99742025-04-30T06:13:27.711ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
99752025-04-30T06:13:27.711ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 481.032175ms iteration = 34 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
99762025-04-30T06:13:27.711ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 35 reason = Signaled
99772025-04-30T06:13:27.711ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
99782025-04-30T06:13:27.711ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
99792025-04-30T06:13:27.711ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99802025-04-30T06:13:27.711ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99812025-04-30T06:13:27.712ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99822025-04-30T06:13:27.716ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
99832025-04-30T06:13:27.716ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
99842025-04-30T06:13:27.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99852025-04-30T06:13:27.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99862025-04-30T06:13:27.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99872025-04-30T06:13:27.720ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99882025-04-30T06:13:27.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99892025-04-30T06:13:27.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
99902025-04-30T06:13:27.722ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
99912025-04-30T06:13:27.722ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
99922025-04-30T06:13:27.722ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
99932025-04-30T06:13:27.723ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
99942025-04-30T06:13:27.726ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
99952025-04-30T06:13:27.727ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
99962025-04-30T06:13:27.728ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: e8 67 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .g..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55693 req_id = 724d818f-b811-4665-a849-795ae65e3016
99972025-04-30T06:13:27.728ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59495,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55693 req_id = 724d818f-b811-4665-a849-795ae65e3016
99982025-04-30T06:13:27.729ZDEBGomicron-dev (store): query key key = _mgs._tcp
99992025-04-30T06:13:27.729ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
100002025-04-30T06:13:27.729ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55693 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 724d818f-b811-4665-a849-795ae65e3016
100012025-04-30T06:13:27.729ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2472, tv_nsec: 528802175 } })
100022025-04-30T06:13:27.730ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
100032025-04-30T06:13:27.730ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 8a aa 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52122 req_id = 62a68502-7bbd-4a5a-b7a0-aec8205b8a84
100042025-04-30T06:13:27.730ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 35498,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52122 req_id = 62a68502-7bbd-4a5a-b7a0-aec8205b8a84
100052025-04-30T06:13:27.731ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
100062025-04-30T06:13:27.731ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52122 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 62a68502-7bbd-4a5a-b7a0-aec8205b8a84
100072025-04-30T06:13:27.768ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
100082025-04-30T06:13:27.768ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100092025-04-30T06:13:27.768ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100102025-04-30T06:13:27.768ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
100112025-04-30T06:13:27.770ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100122025-04-30T06:13:27.773ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100132025-04-30T06:13:27.785ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
100142025-04-30T06:13:27.785ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100152025-04-30T06:13:27.785ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
100162025-04-30T06:13:27.786ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100172025-04-30T06:13:27.788ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
100182025-04-30T06:13:27.791ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 95 d6 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62899 req_id = 87ab3f42-603b-4208-a172-9b932169175a
100192025-04-30T06:13:27.791ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38358,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62899 req_id = 87ab3f42-603b-4208-a172-9b932169175a
100202025-04-30T06:13:27.791ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100212025-04-30T06:13:27.791ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
100222025-04-30T06:13:27.791ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
100232025-04-30T06:13:27.791ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 38358, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62899 req_id = 87ab3f42-603b-4208-a172-9b932169175a
100242025-04-30T06:13:27.791ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
100252025-04-30T06:13:27.792ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
100262025-04-30T06:13:27.792ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
100272025-04-30T06:13:27.792ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100282025-04-30T06:13:27.796ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "52b9c073-ba54-4f58-b27f-57a4999cc223", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100292025-04-30T06:13:27.797ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
100302025-04-30T06:13:27.800ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b67b71d9-a125-4e5e-af89-e899a4f875c0", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100312025-04-30T06:13:27.801ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
100322025-04-30T06:13:27.805ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc33bc7c-d4c6-485e-9095-9a9d8be4a676", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100332025-04-30T06:13:27.805ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
100342025-04-30T06:13:27.809ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7644468b-edb8-41c8-a0f4-30e3d70685a8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100352025-04-30T06:13:27.809ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
100362025-04-30T06:13:27.818ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bec4876f-2132-4cf5-8ce5-bda2654a1912", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100372025-04-30T06:13:27.819ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
100382025-04-30T06:13:27.830ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0161d29-f6d5-4864-a397-0cf974612171", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100392025-04-30T06:13:27.831ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
100402025-04-30T06:13:27.835ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f673708-65ef-40c2-b0dd-9b2b1fe813dc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100412025-04-30T06:13:27.837ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
100422025-04-30T06:13:27.840ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5c62878b-8653-4d63-831f-3dccb2003e7b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100432025-04-30T06:13:27.841ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
100442025-04-30T06:13:27.842ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f9c75b6-cd01-479d-81c5-32359e41f677", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100452025-04-30T06:13:27.842ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
100462025-04-30T06:13:27.844ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3357b9d7-74af-46df-af93-1fedfe9b0a80", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100472025-04-30T06:13:27.844ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
100482025-04-30T06:13:27.847ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed5c5ff9-bcde-4426-9fc7-04a38fd31a2e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100492025-04-30T06:13:27.847ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
100502025-04-30T06:13:27.848ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c327059c-f7e7-455b-a790-f9f3727050a2", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100512025-04-30T06:13:27.848ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
100522025-04-30T06:13:27.850ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26ae8766-b0a5-4e0d-ba11-a67431150189", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100532025-04-30T06:13:27.850ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
100542025-04-30T06:13:27.853ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd0e5fb7-9f71-4acc-8006-a6ee06b438ff", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100552025-04-30T06:13:27.853ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
100562025-04-30T06:13:27.855ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "30e40913-4d65-4eeb-82c0-0007e9027114", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100572025-04-30T06:13:27.855ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
100582025-04-30T06:13:27.857ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
100592025-04-30T06:13:27.858ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6e74086-f6c1-44c9-bb1f-3d2dbd8cc37d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100602025-04-30T06:13:27.858ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
100612025-04-30T06:13:27.861ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ef78487-aca8-4125-9202-d611d0a644e2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100622025-04-30T06:13:27.862ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
100632025-04-30T06:13:27.863ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cb9277e-f7f0-49d1-9bb7-f19cde537806", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100642025-04-30T06:13:27.871ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
100652025-04-30T06:13:27.876ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c8ff96dd-aaf0-40d1-acef-1ec87d67dc1b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100662025-04-30T06:13:27.876ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
100672025-04-30T06:13:27.877ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "62e1088f-2321-4de6-8ba5-1424253964c3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100682025-04-30T06:13:27.877ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
100692025-04-30T06:13:27.881ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "23ca2987-3623-499e-955a-ffc1346f398f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100702025-04-30T06:13:27.882ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
100712025-04-30T06:13:27.883ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "34cab3b8-3e63-4933-839b-6b9e5f839eef", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100722025-04-30T06:13:27.884ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
100732025-04-30T06:13:27.888ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19871e9e-af3a-4932-84b3-c6983de2ef9e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100742025-04-30T06:13:27.888ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
100752025-04-30T06:13:27.894ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7772e6ec-ea4d-4166-884f-f7e1f61f3608", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100762025-04-30T06:13:27.894ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
100772025-04-30T06:13:27.896ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffeef1c0-c01e-48a3-89fc-faa29cd6e645", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100782025-04-30T06:13:27.897ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
100792025-04-30T06:13:27.899ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f835a5e-9b48-490e-8c2a-2e25f3e33309", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100802025-04-30T06:13:27.899ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
100812025-04-30T06:13:27.902ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "02f2444a-4189-4683-bd7f-d8df84bd44e2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100822025-04-30T06:13:27.902ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
100832025-04-30T06:13:27.904ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5e69e67-8c78-4de3-91bf-9c370eeb58fe", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100842025-04-30T06:13:27.905ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
100852025-04-30T06:13:27.907ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbd73d60-726c-4495-9820-a38a71798c62", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100862025-04-30T06:13:27.907ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
100872025-04-30T06:13:27.908ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
100882025-04-30T06:13:27.908ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
100892025-04-30T06:13:27.908ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
100902025-04-30T06:13:27.908ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
100912025-04-30T06:13:27.909ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
100922025-04-30T06:13:27.911ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ba88efb-32f0-42a9-b1ba-3a57312c1a72", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100932025-04-30T06:13:27.911ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
100942025-04-30T06:13:27.912ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8142d5d8-8ff2-4b84-9717-79b99e09c1cd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100952025-04-30T06:13:27.912ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
100962025-04-30T06:13:27.912ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
100972025-04-30T06:13:27.916ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac104700-947b-4c83-86b4-dc1d55c6fbc8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
100982025-04-30T06:13:27.916ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
100992025-04-30T06:13:27.917ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cb13adf-2d6d-43af-aea2-17baee09c0ad", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101002025-04-30T06:13:27.917ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
101012025-04-30T06:13:27.918ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
101022025-04-30T06:13:27.918ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101032025-04-30T06:13:27.918ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
101042025-04-30T06:13:27.918ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b440a53f-5baf-419d-bae2-c25dbda9436a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101052025-04-30T06:13:27.918ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
101062025-04-30T06:13:27.919ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101072025-04-30T06:13:27.920ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92b9c21c-cb7f-424e-8b54-bd867b803874", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101082025-04-30T06:13:27.920ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
101092025-04-30T06:13:27.921ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101102025-04-30T06:13:27.921ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
101112025-04-30T06:13:27.922ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101122025-04-30T06:13:27.924ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03335b9f-db4c-47d7-a91b-40ebb1c714ff", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101132025-04-30T06:13:27.925ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
101142025-04-30T06:13:27.932ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "583ec085-98df-4366-93b9-2dfc2e001793", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101152025-04-30T06:13:27.932ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
101162025-04-30T06:13:27.935ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9436916d-aa4a-4dd1-b848-d4f7a422403e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101172025-04-30T06:13:27.935ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
101182025-04-30T06:13:27.938ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c2c6ae5-0e82-46d5-8ec9-6eb8a765d5be", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101192025-04-30T06:13:27.939ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
101202025-04-30T06:13:27.942ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "24549eab-31a5-470f-ae64-cdb8ae918694", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101212025-04-30T06:13:27.943ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
101222025-04-30T06:13:27.945ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca7a0aad-fed9-487b-8dc7-366113c79ed2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101232025-04-30T06:13:27.946ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
101242025-04-30T06:13:27.947ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a96b8ff-7abf-4374-ba32-2c19c4d94a70", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101252025-04-30T06:13:27.947ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
101262025-04-30T06:13:27.949ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30891845-1149-4833-98c8-7e25836d790e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101272025-04-30T06:13:27.949ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
101282025-04-30T06:13:27.950ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e1ad302-0029-48d7-8fa8-74218b896a61", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101292025-04-30T06:13:27.950ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
101302025-04-30T06:13:27.951ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe302e15-5c5f-4d25-843d-9bda8371215e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101312025-04-30T06:13:27.951ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
101322025-04-30T06:13:27.952ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101332025-04-30T06:13:27.952ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101342025-04-30T06:13:27.952ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101352025-04-30T06:13:27.952ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101362025-04-30T06:13:27.958ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
101372025-04-30T06:13:27.958ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101382025-04-30T06:13:27.963ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
101392025-04-30T06:13:27.963ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
101402025-04-30T06:13:27.963ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101412025-04-30T06:13:27.963ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101422025-04-30T06:13:27.963ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101432025-04-30T06:13:27.963ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101442025-04-30T06:13:27.966ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101452025-04-30T06:13:27.973ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
101462025-04-30T06:13:27.973ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101472025-04-30T06:13:27.973ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101482025-04-30T06:13:27.974ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101492025-04-30T06:13:28.012ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
101502025-04-30T06:13:28.012ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101512025-04-30T06:13:28.012ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101522025-04-30T06:13:28.012ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
101532025-04-30T06:13:28.013ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101542025-04-30T06:13:28.057ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
101552025-04-30T06:13:28.058ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
101562025-04-30T06:13:28.058ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:62083 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
101572025-04-30T06:13:28.059ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:62083 req_id = 9657b800-82e6-4d08-89c6-3a2facc88312 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
101582025-04-30T06:13:28.060ZINFOomicron-dev (dropshot): request completed latency_us = 780 local_addr = [::1]:61881 method = GET remote_addr = [::1]:62083 req_id = 9657b800-82e6-4d08-89c6-3a2facc88312 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
101592025-04-30T06:13:28.060ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "9657b800-82e6-4d08-89c6-3a2facc88312", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
101602025-04-30T06:13:28.062ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101612025-04-30T06:13:28.064ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
101622025-04-30T06:13:28.064ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
101632025-04-30T06:13:28.064ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
101642025-04-30T06:13:28.067ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
101652025-04-30T06:13:28.067ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101662025-04-30T06:13:28.067ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101672025-04-30T06:13:28.067ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101682025-04-30T06:13:28.068ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101692025-04-30T06:13:28.070ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
101702025-04-30T06:13:28.070ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101712025-04-30T06:13:28.070ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
101722025-04-30T06:13:28.071ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101732025-04-30T06:13:28.074ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101742025-04-30T06:13:28.075ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
101752025-04-30T06:13:28.075ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
101762025-04-30T06:13:28.076ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
101772025-04-30T06:13:28.078ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
101782025-04-30T06:13:28.079ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101792025-04-30T06:13:28.079ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101802025-04-30T06:13:28.080ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101812025-04-30T06:13:28.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
101822025-04-30T06:13:28.089ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
101832025-04-30T06:13:28.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
101842025-04-30T06:13:28.090ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
101852025-04-30T06:13:28.095ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
101862025-04-30T06:13:28.095ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
101872025-04-30T06:13:28.099ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
101882025-04-30T06:13:28.099ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
101892025-04-30T06:13:28.134ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
101902025-04-30T06:13:28.134ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
101912025-04-30T06:13:28.135ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 423.091251ms iteration = 35 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
101922025-04-30T06:13:28.135ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 36 reason = Signaled
101932025-04-30T06:13:28.135ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
101942025-04-30T06:13:28.135ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
101952025-04-30T06:13:28.135ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
101962025-04-30T06:13:28.135ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
101972025-04-30T06:13:28.135ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
101982025-04-30T06:13:28.138ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
101992025-04-30T06:13:28.141ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
102002025-04-30T06:13:28.143ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
102012025-04-30T06:13:28.143ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
102022025-04-30T06:13:28.143ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
102032025-04-30T06:13:28.144ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
102042025-04-30T06:13:28.148ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
102052025-04-30T06:13:28.148ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
102062025-04-30T06:13:28.148ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 9f 03 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61089 req_id = 8577f978-f8fa-4023-b615-92d280067dff
102072025-04-30T06:13:28.148ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 40707,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61089 req_id = 8577f978-f8fa-4023-b615-92d280067dff
102082025-04-30T06:13:28.148ZDEBGomicron-dev (store): query key key = _mgs._tcp
102092025-04-30T06:13:28.148ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
102102025-04-30T06:13:28.148ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61089 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8577f978-f8fa-4023-b615-92d280067dff
102112025-04-30T06:13:28.149ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2472, tv_nsec: 948184276 } })
102122025-04-30T06:13:28.149ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
102132025-04-30T06:13:28.152ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f9 eb 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52407 req_id = 31ddf097-6833-421c-84db-1c6b8cbe94ce
102142025-04-30T06:13:28.152ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63979,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52407 req_id = 31ddf097-6833-421c-84db-1c6b8cbe94ce
102152025-04-30T06:13:28.153ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
102162025-04-30T06:13:28.153ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52407 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 31ddf097-6833-421c-84db-1c6b8cbe94ce
102172025-04-30T06:13:28.190ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
102182025-04-30T06:13:28.191ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
102192025-04-30T06:13:28.191ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 6d cf 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c m............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62438 req_id = 8c397f27-ed57-456e-a307-7cf40e3aa919
102202025-04-30T06:13:28.191ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28111,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62438 req_id = 8c397f27-ed57-456e-a307-7cf40e3aa919
102212025-04-30T06:13:28.191ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
102222025-04-30T06:13:28.191ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 28111, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62438 req_id = 8c397f27-ed57-456e-a307-7cf40e3aa919
102232025-04-30T06:13:28.192ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
102242025-04-30T06:13:28.192ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
102252025-04-30T06:13:28.193ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
102262025-04-30T06:13:28.193ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
102272025-04-30T06:13:28.193ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102282025-04-30T06:13:28.193ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
102292025-04-30T06:13:28.193ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
102302025-04-30T06:13:28.193ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102312025-04-30T06:13:28.194ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "79e2b542-2cbd-4711-aebd-1eeb4cf1ff1e", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102322025-04-30T06:13:28.195ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
102332025-04-30T06:13:28.196ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "873a3905-46ef-4b2c-9242-a3b8020c9b56", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102342025-04-30T06:13:28.196ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
102352025-04-30T06:13:28.198ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
102362025-04-30T06:13:28.199ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8448a4d-8950-4086-a531-449d2df94389", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102372025-04-30T06:13:28.199ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
102382025-04-30T06:13:28.202ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "52bfcb6b-e9bd-4d28-8dfa-c81f290effef", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102392025-04-30T06:13:28.202ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
102402025-04-30T06:13:28.204ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "18405595-db42-4b5b-ab7f-56831eada5d6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102412025-04-30T06:13:28.204ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
102422025-04-30T06:13:28.205ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
102432025-04-30T06:13:28.206ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102442025-04-30T06:13:28.206ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80d2a552-1f8c-4635-bc5e-4ba0d82c1b46", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102452025-04-30T06:13:28.206ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
102462025-04-30T06:13:28.206ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
102472025-04-30T06:13:28.206ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102482025-04-30T06:13:28.209ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
102492025-04-30T06:13:28.209ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
102502025-04-30T06:13:28.209ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "58f3fcaa-d3e9-4799-8ce7-ea8efe2aab55", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102512025-04-30T06:13:28.210ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
102522025-04-30T06:13:28.210ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
102532025-04-30T06:13:28.212ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "82f73c8e-52df-4564-9683-ba246e283e8c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102542025-04-30T06:13:28.213ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
102552025-04-30T06:13:28.213ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73663866-8d5f-44a0-9181-7698471e5ea6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102562025-04-30T06:13:28.213ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
102572025-04-30T06:13:28.215ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e37c171-0ce0-49ec-a3c1-03ac0b614ea9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102582025-04-30T06:13:28.215ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
102592025-04-30T06:13:28.217ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "01dd0092-368f-45e9-8ad4-3895c310171d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102602025-04-30T06:13:28.217ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
102612025-04-30T06:13:28.219ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4678620-cc66-4c76-8bd6-b7c0a1bd35e8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102622025-04-30T06:13:28.220ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
102632025-04-30T06:13:28.223ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23b003e6-d443-44a5-8d49-4e93cd0c4643", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102642025-04-30T06:13:28.223ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
102652025-04-30T06:13:28.229ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62984949-51aa-40ac-a3cc-1400241642bf", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102662025-04-30T06:13:28.230ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
102672025-04-30T06:13:28.235ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b810ce8-5ef1-4225-8e1b-8818ddf6d051", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102682025-04-30T06:13:28.235ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
102692025-04-30T06:13:28.238ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f119d5e3-2b7f-4bd6-a109-4fa85f6c13c2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102702025-04-30T06:13:28.238ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
102712025-04-30T06:13:28.243ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "086857d7-8acf-4fed-9b86-e5bc0cee03ec", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102722025-04-30T06:13:28.243ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
102732025-04-30T06:13:28.246ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85a287a2-96c6-455a-bce1-97f0d3b1b118", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102742025-04-30T06:13:28.246ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
102752025-04-30T06:13:28.248ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
102762025-04-30T06:13:28.250ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e9241d1-5fc7-4d6c-b7ec-1e858b36e899", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102772025-04-30T06:13:28.251ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
102782025-04-30T06:13:28.252ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7eb5227d-82b3-4ee8-a62b-95d0f88ba865", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102792025-04-30T06:13:28.252ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
102802025-04-30T06:13:28.254ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0703e78f-154d-4a31-8475-4bdca4b8fc7f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102812025-04-30T06:13:28.254ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
102822025-04-30T06:13:28.257ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bba8c750-083a-4831-9d9e-bc35d7086ffa", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102832025-04-30T06:13:28.257ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
102842025-04-30T06:13:28.259ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "59f19710-c3f1-46ff-8af6-3845d9a14a20", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102852025-04-30T06:13:28.259ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
102862025-04-30T06:13:28.264ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfc66751-1a35-4355-984c-abe16c255a37", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102872025-04-30T06:13:28.264ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
102882025-04-30T06:13:28.272ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e2468e1-4ba6-4a84-b1e4-8f1f387f24c2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102892025-04-30T06:13:28.272ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
102902025-04-30T06:13:28.274ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ca09004-a853-4067-ba6b-7a5c9f85861c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102912025-04-30T06:13:28.275ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
102922025-04-30T06:13:28.277ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "86517458-281f-462f-803b-7b1a93190fe7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102932025-04-30T06:13:28.279ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
102942025-04-30T06:13:28.282ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ab23c8e-0285-4275-8a14-349175b6bd82", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102952025-04-30T06:13:28.282ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
102962025-04-30T06:13:28.284ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "356580f5-a757-4671-ae2d-f7eb84530c0d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102972025-04-30T06:13:28.285ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
102982025-04-30T06:13:28.287ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "646de5b3-9021-48c8-9117-3649b0d511a1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
102992025-04-30T06:13:28.287ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
103002025-04-30T06:13:28.290ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31e0a016-aeb1-4aad-85d9-a2d47630c19c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103012025-04-30T06:13:28.290ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
103022025-04-30T06:13:28.293ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "132d5228-9b93-45a0-8a8a-b28262055141", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103032025-04-30T06:13:28.293ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
103042025-04-30T06:13:28.296ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae4486b3-566f-4a06-ac5e-998900f02fed", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103052025-04-30T06:13:28.297ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
103062025-04-30T06:13:28.299ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
103072025-04-30T06:13:28.299ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103082025-04-30T06:13:28.299ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
103092025-04-30T06:13:28.299ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
103102025-04-30T06:13:28.299ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeb8310e-23df-4b4c-b8cf-86d68460e727", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103112025-04-30T06:13:28.299ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
103122025-04-30T06:13:28.300ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103132025-04-30T06:13:28.306ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "15d9a47f-d87f-4574-8b3d-9a2f3fdf8d9a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103142025-04-30T06:13:28.306ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
103152025-04-30T06:13:28.311ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103162025-04-30T06:13:28.315ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "768a604a-b9c3-489b-9f60-8c2fe81b3f8f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103172025-04-30T06:13:28.316ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
103182025-04-30T06:13:28.316ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
103192025-04-30T06:13:28.316ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
103202025-04-30T06:13:28.316ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
103212025-04-30T06:13:28.317ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103222025-04-30T06:13:28.320ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
103232025-04-30T06:13:28.320ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
103242025-04-30T06:13:28.320ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103252025-04-30T06:13:28.321ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4223fc7-8653-48a5-a8f3-56faae602573", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103262025-04-30T06:13:28.322ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
103272025-04-30T06:13:28.324ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "579f5f61-9686-421f-b6e2-5993a89c578b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103282025-04-30T06:13:28.325ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
103292025-04-30T06:13:28.328ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b072d6c9-18ef-4df2-a215-7bd194dd26e6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103302025-04-30T06:13:28.329ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
103312025-04-30T06:13:28.334ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3259b0c5-9d45-4ad1-817c-e00a20aa3eb8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103322025-04-30T06:13:28.339ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
103332025-04-30T06:13:28.348ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "893c5e11-9a3e-4b97-a8df-afdcf703fc12", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103342025-04-30T06:13:28.348ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
103352025-04-30T06:13:28.373ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c4600bf-26c6-4980-adca-8968db85a6b4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103362025-04-30T06:13:28.373ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
103372025-04-30T06:13:28.376ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3eb5d383-8ba4-4d15-b37d-dd09f7a423f9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103382025-04-30T06:13:28.376ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
103392025-04-30T06:13:28.378ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b31d8b36-c0cb-44a8-bd37-24499f9df30c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103402025-04-30T06:13:28.378ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
103412025-04-30T06:13:28.380ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
103422025-04-30T06:13:28.380ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c65d527-462e-45fd-8cd6-83f59de11c58", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103432025-04-30T06:13:28.380ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
103442025-04-30T06:13:28.380ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103452025-04-30T06:13:28.380ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103462025-04-30T06:13:28.380ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103472025-04-30T06:13:28.381ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103482025-04-30T06:13:28.384ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103492025-04-30T06:13:28.391ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
103502025-04-30T06:13:28.391ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
103512025-04-30T06:13:28.391ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103522025-04-30T06:13:28.391ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103532025-04-30T06:13:28.391ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103542025-04-30T06:13:28.392ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103552025-04-30T06:13:28.396ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103562025-04-30T06:13:28.402ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
103572025-04-30T06:13:28.402ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103582025-04-30T06:13:28.402ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103592025-04-30T06:13:28.403ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103602025-04-30T06:13:28.431ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
103612025-04-30T06:13:28.432ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103622025-04-30T06:13:28.432ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
103632025-04-30T06:13:28.432ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
103642025-04-30T06:13:28.433ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103652025-04-30T06:13:28.458ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
103662025-04-30T06:13:28.459ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
103672025-04-30T06:13:28.460ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:37469 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
103682025-04-30T06:13:28.460ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:37469 req_id = e743042e-82e6-445b-b08b-5978fb9462fe sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
103692025-04-30T06:13:28.462ZINFOomicron-dev (dropshot): request completed latency_us = 1153 local_addr = [::1]:61881 method = GET remote_addr = [::1]:37469 req_id = e743042e-82e6-445b-b08b-5978fb9462fe response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
103702025-04-30T06:13:28.462ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103712025-04-30T06:13:28.462ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e743042e-82e6-445b-b08b-5978fb9462fe", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
103722025-04-30T06:13:28.467ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
103732025-04-30T06:13:28.468ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
103742025-04-30T06:13:28.468ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
103752025-04-30T06:13:28.472ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
103762025-04-30T06:13:28.472ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
103772025-04-30T06:13:28.472ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
103782025-04-30T06:13:28.472ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
103792025-04-30T06:13:28.472ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103802025-04-30T06:13:28.473ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103812025-04-30T06:13:28.473ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103822025-04-30T06:13:28.473ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103832025-04-30T06:13:28.474ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103842025-04-30T06:13:28.478ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
103852025-04-30T06:13:28.478ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
103862025-04-30T06:13:28.480ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
103872025-04-30T06:13:28.481ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
103882025-04-30T06:13:28.517ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
103892025-04-30T06:13:28.517ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
103902025-04-30T06:13:28.517ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
103912025-04-30T06:13:28.518ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
103922025-04-30T06:13:28.547ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
103932025-04-30T06:13:28.553ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
103942025-04-30T06:13:28.553ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
103952025-04-30T06:13:28.553ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 418.807978ms iteration = 36 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
103962025-04-30T06:13:28.553ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 37 reason = Signaled
103972025-04-30T06:13:28.553ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
103982025-04-30T06:13:28.553ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
103992025-04-30T06:13:28.553ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
104002025-04-30T06:13:28.553ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
104012025-04-30T06:13:28.554ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
104022025-04-30T06:13:28.556ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104032025-04-30T06:13:28.561ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
104042025-04-30T06:13:28.561ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
104052025-04-30T06:13:28.561ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
104062025-04-30T06:13:28.561ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
104072025-04-30T06:13:28.565ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
104082025-04-30T06:13:28.565ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
104092025-04-30T06:13:28.565ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 1f f8 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:53550 req_id = 9f1a1fc5-f987-4992-ba42-a03590d87a92
104102025-04-30T06:13:28.565ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8184,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53550 req_id = 9f1a1fc5-f987-4992-ba42-a03590d87a92
104112025-04-30T06:13:28.566ZDEBGomicron-dev (store): query key key = _mgs._tcp
104122025-04-30T06:13:28.566ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
104132025-04-30T06:13:28.566ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53550 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 9f1a1fc5-f987-4992-ba42-a03590d87a92
104142025-04-30T06:13:28.566ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2473, tv_nsec: 365742884 } })
104152025-04-30T06:13:28.566ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
104162025-04-30T06:13:28.567ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 47 84 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e G...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52519 req_id = d8bdf96b-9c95-4e44-9823-05ccf9686f44
104172025-04-30T06:13:28.567ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 18308,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52519 req_id = d8bdf96b-9c95-4e44-9823-05ccf9686f44
104182025-04-30T06:13:28.567ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
104192025-04-30T06:13:28.567ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52519 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d8bdf96b-9c95-4e44-9823-05ccf9686f44
104202025-04-30T06:13:28.639ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
104212025-04-30T06:13:28.639ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
104222025-04-30T06:13:28.639ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
104232025-04-30T06:13:28.639ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
104242025-04-30T06:13:28.639ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
104252025-04-30T06:13:28.640ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
104262025-04-30T06:13:28.644ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 16 96 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54935 req_id = 49ae7fab-43ab-47a6-b2d8-88fc1f0f0900
104272025-04-30T06:13:28.644ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5782,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54935 req_id = 49ae7fab-43ab-47a6-b2d8-88fc1f0f0900
104282025-04-30T06:13:28.644ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
104292025-04-30T06:13:28.644ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5782, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54935 req_id = 49ae7fab-43ab-47a6-b2d8-88fc1f0f0900
104302025-04-30T06:13:28.647ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
104312025-04-30T06:13:28.647ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
104322025-04-30T06:13:28.647ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
104332025-04-30T06:13:28.649ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
104342025-04-30T06:13:28.654ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
104352025-04-30T06:13:28.654ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
104362025-04-30T06:13:28.654ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
104372025-04-30T06:13:28.654ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
104382025-04-30T06:13:28.654ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "7435a15b-c226-4e88-bf55-3637dc8fc83e", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
104392025-04-30T06:13:28.655ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
104402025-04-30T06:13:28.657ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e73d3ab7-a4aa-4f32-9806-c767fde1ff39", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104412025-04-30T06:13:28.657ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
104422025-04-30T06:13:28.657ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
104432025-04-30T06:13:28.657ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
104442025-04-30T06:13:28.658ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
104452025-04-30T06:13:28.659ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "78566494-ab49-4e30-9638-1cef64606b3a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104462025-04-30T06:13:28.659ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
104472025-04-30T06:13:28.662ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "37d45e3f-c197-4d99-b843-70cf79ac8570", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
104482025-04-30T06:13:28.662ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
104492025-04-30T06:13:28.667ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "df75682a-fabd-4f91-ba34-64b1eae99bc0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
104502025-04-30T06:13:28.667ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
104512025-04-30T06:13:28.668ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b830a68b-272f-45ad-8504-c79900b5e033", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
104522025-04-30T06:13:28.670ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
104532025-04-30T06:13:28.682ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab2433ed-3c50-404b-89d8-2d5309050061", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
104542025-04-30T06:13:28.682ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
104552025-04-30T06:13:28.682ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
104562025-04-30T06:13:28.683ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
104572025-04-30T06:13:28.683ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3e8a8faa-cee9-44f7-a84d-d515adc2069e uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
104582025-04-30T06:13:28.683ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3e8a8faa-cee9-44f7-a84d-d515adc2069e resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
104592025-04-30T06:13:28.683ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3e8a8faa-cee9-44f7-a84d-d515adc2069e roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
104602025-04-30T06:13:28.684ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3e8a8faa-cee9-44f7-a84d-d515adc2069e resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
104612025-04-30T06:13:28.685ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae0d7964-53d0-4e99-bf42-30bce2a8b14e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104622025-04-30T06:13:28.685ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
104632025-04-30T06:13:28.686ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f0f5e42-a042-4d0d-8cd9-40c61dc2efcb", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104642025-04-30T06:13:28.687ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
104652025-04-30T06:13:28.688ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d801844b-55d2-4f40-a30a-092b3da2341c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104662025-04-30T06:13:28.688ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
104672025-04-30T06:13:28.689ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "945c90ff-2e04-418a-a32a-29abcb592d10", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104682025-04-30T06:13:28.689ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
104692025-04-30T06:13:28.690ZINFOomicron-dev (dropshot_internal): request completed latency_us = 6889 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3e8a8faa-cee9-44f7-a84d-d515adc2069e response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
104702025-04-30T06:13:28.690ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e8a8faa-cee9-44f7-a84d-d515adc2069e", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104712025-04-30T06:13:28.690ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
104722025-04-30T06:13:28.691ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = df100ba6-c722-4c5e-b645-91f85f6ce133 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
104732025-04-30T06:13:28.691ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = df100ba6-c722-4c5e-b645-91f85f6ce133 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
104742025-04-30T06:13:28.691ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = df100ba6-c722-4c5e-b645-91f85f6ce133 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
104752025-04-30T06:13:28.692ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = df100ba6-c722-4c5e-b645-91f85f6ce133 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
104762025-04-30T06:13:28.692ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4800811-570d-4f33-aae5-4dbb707e8218", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104772025-04-30T06:13:28.692ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
104782025-04-30T06:13:28.694ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "160a3fb5-0fd5-420c-844f-71a8dc25de01", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104792025-04-30T06:13:28.694ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
104802025-04-30T06:13:28.695ZINFOomicron-dev (dropshot_internal): request completed latency_us = 3957 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = df100ba6-c722-4c5e-b645-91f85f6ce133 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
104812025-04-30T06:13:28.695ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "df100ba6-c722-4c5e-b645-91f85f6ce133", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104822025-04-30T06:13:28.695ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
104832025-04-30T06:13:28.695ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
104842025-04-30T06:13:28.695ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
104852025-04-30T06:13:28.698ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "399f60c0-a962-401a-a7be-9094c0754559", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104862025-04-30T06:13:28.698ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
104872025-04-30T06:13:28.700ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "192b2679-fc2b-48f3-acfe-5a6fd80d39a5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104882025-04-30T06:13:28.700ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
104892025-04-30T06:13:28.702ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "62cfec5b-9697-4690-a746-7e9eed1181d5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104902025-04-30T06:13:28.702ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
104912025-04-30T06:13:28.704ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c0a8c8b-3ec8-439c-995d-b16b9121d170", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104922025-04-30T06:13:28.704ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
104932025-04-30T06:13:28.706ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7993333-c562-4533-958e-b84fbc6af303", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104942025-04-30T06:13:28.706ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
104952025-04-30T06:13:28.707ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "438f90de-539a-440e-8553-6c86ab4da358", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104962025-04-30T06:13:28.708ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
104972025-04-30T06:13:28.708ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1af88a5-87f4-4989-a0e6-cc61ae505d87", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
104982025-04-30T06:13:28.709ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
104992025-04-30T06:13:28.710ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "07577cc4-816b-44ab-9ee4-97d5449fcc70", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105002025-04-30T06:13:28.710ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
105012025-04-30T06:13:28.711ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2cd64e72-4de7-41a1-b78f-db9008001399", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105022025-04-30T06:13:28.711ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
105032025-04-30T06:13:28.712ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33575570-4095-4cb4-b53c-de97ab4d95b7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105042025-04-30T06:13:28.712ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
105052025-04-30T06:13:28.712ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
105062025-04-30T06:13:28.714ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7334ecfd-3c62-4f5d-9c83-d8a9ee651eb4", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105072025-04-30T06:13:28.714ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
105082025-04-30T06:13:28.717ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea2694f8-3bde-4628-ba68-3c0886beed4a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105092025-04-30T06:13:28.717ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
105102025-04-30T06:13:28.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
105112025-04-30T06:13:28.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
105122025-04-30T06:13:28.719ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "410e3615-271d-4d06-8e05-6ed4fb58a1c9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105132025-04-30T06:13:28.719ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
105142025-04-30T06:13:28.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
105152025-04-30T06:13:28.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
105162025-04-30T06:13:28.722ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
105172025-04-30T06:13:28.722ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
105182025-04-30T06:13:28.726ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9338dc18-9756-404d-9230-a88ea1ac7b50", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105192025-04-30T06:13:28.726ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
105202025-04-30T06:13:28.730ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "205a0c31-b3b6-4164-bf8e-12700355d4f4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105212025-04-30T06:13:28.730ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
105222025-04-30T06:13:28.732ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3133113d-07a2-452f-83ca-f17ffab84c5d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105232025-04-30T06:13:28.733ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
105242025-04-30T06:13:28.735ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fec8c91-b3ae-4b4f-ad22-a9650b5841ff", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105252025-04-30T06:13:28.735ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
105262025-04-30T06:13:28.736ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6640baad-94bc-4f7f-9c5a-b4a11af90092", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105272025-04-30T06:13:28.736ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
105282025-04-30T06:13:28.737ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc54090e-be08-4d2d-b61e-95e40669f2a2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105292025-04-30T06:13:28.737ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
105302025-04-30T06:13:28.739ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6f80433-974b-40f3-9f46-10536f34937e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105312025-04-30T06:13:28.740ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
105322025-04-30T06:13:28.741ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce46c8d7-eb85-4084-85a1-9ff4052772df", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105332025-04-30T06:13:28.741ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
105342025-04-30T06:13:28.743ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9dcba2a0-4ade-4686-98c6-8411437961b0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105352025-04-30T06:13:28.743ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
105362025-04-30T06:13:28.752ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccdeb816-b950-4d0e-856c-0533cfda1a9d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105372025-04-30T06:13:28.753ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
105382025-04-30T06:13:28.757ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "59fe51ee-9427-4cd0-ae57-b35db6bdc98c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105392025-04-30T06:13:28.757ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
105402025-04-30T06:13:28.760ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9628190b-1c1e-4606-af32-a4bb2706131c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105412025-04-30T06:13:28.760ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
105422025-04-30T06:13:28.763ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0de6b72c-e093-4c42-8de3-6cbe834965ff", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105432025-04-30T06:13:28.763ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
105442025-04-30T06:13:28.763ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
105452025-04-30T06:13:28.763ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105462025-04-30T06:13:28.763ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105472025-04-30T06:13:28.763ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
105482025-04-30T06:13:28.764ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105492025-04-30T06:13:28.767ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8db9604-9964-4b1f-a434-da107fc51864", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105502025-04-30T06:13:28.767ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
105512025-04-30T06:13:28.771ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105522025-04-30T06:13:28.772ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "231ff303-e608-4f26-a7ed-f0af81e9da30", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:27 GMT"} })
105532025-04-30T06:13:28.772ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
105542025-04-30T06:13:28.773ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4526c72-df6a-4595-ab61-7421115dbb4b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105552025-04-30T06:13:28.774ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
105562025-04-30T06:13:28.775ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ed1bdcf-84cf-44e8-b3d5-ae838246dc49", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105572025-04-30T06:13:28.775ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
105582025-04-30T06:13:28.776ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0f35c33-427a-4201-8cb8-a081904f3f1c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105592025-04-30T06:13:28.776ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
105602025-04-30T06:13:28.777ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "64e2626b-f126-4867-816c-ed3c493841e8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105612025-04-30T06:13:28.777ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
105622025-04-30T06:13:28.778ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105632025-04-30T06:13:28.778ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105642025-04-30T06:13:28.778ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105652025-04-30T06:13:28.778ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105662025-04-30T06:13:28.780ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
105672025-04-30T06:13:28.780ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105682025-04-30T06:13:28.780ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
105692025-04-30T06:13:28.780ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105702025-04-30T06:13:28.781ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105712025-04-30T06:13:28.785ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
105722025-04-30T06:13:28.785ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
105732025-04-30T06:13:28.785ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105742025-04-30T06:13:28.785ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105752025-04-30T06:13:28.785ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105762025-04-30T06:13:28.785ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105772025-04-30T06:13:28.787ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
105782025-04-30T06:13:28.787ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
105792025-04-30T06:13:28.787ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
105802025-04-30T06:13:28.788ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
105812025-04-30T06:13:28.793ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
105822025-04-30T06:13:28.793ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105832025-04-30T06:13:28.793ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105842025-04-30T06:13:28.794ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
105852025-04-30T06:13:28.833ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
105862025-04-30T06:13:28.858ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
105872025-04-30T06:13:28.859ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
105882025-04-30T06:13:28.859ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:40697 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
105892025-04-30T06:13:28.860ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:40697 req_id = a9e0d4e6-8ac4-46cf-abd1-50538569db17 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
105902025-04-30T06:13:28.860ZINFOomicron-dev (dropshot): request completed latency_us = 677 local_addr = [::1]:61881 method = GET remote_addr = [::1]:40697 req_id = a9e0d4e6-8ac4-46cf-abd1-50538569db17 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
105912025-04-30T06:13:28.861ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9e0d4e6-8ac4-46cf-abd1-50538569db17", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
105922025-04-30T06:13:28.866ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
105932025-04-30T06:13:28.866ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
105942025-04-30T06:13:28.866ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
105952025-04-30T06:13:28.868ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
105962025-04-30T06:13:28.869ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
105972025-04-30T06:13:28.869ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
105982025-04-30T06:13:28.869ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
105992025-04-30T06:13:28.869ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
106002025-04-30T06:13:28.872ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106012025-04-30T06:13:28.878ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
106022025-04-30T06:13:28.878ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
106032025-04-30T06:13:28.878ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
106042025-04-30T06:13:28.879ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
106052025-04-30T06:13:28.884ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
106062025-04-30T06:13:28.884ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
106072025-04-30T06:13:28.884ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106082025-04-30T06:13:28.884ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
106092025-04-30T06:13:28.886ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
106102025-04-30T06:13:28.889ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106112025-04-30T06:13:28.897ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
106122025-04-30T06:13:28.897ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106132025-04-30T06:13:28.897ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
106142025-04-30T06:13:28.897ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
106152025-04-30T06:13:28.907ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
106162025-04-30T06:13:28.907ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
106172025-04-30T06:13:28.907ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 354.177656ms iteration = 37 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
106182025-04-30T06:13:28.907ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 38 reason = Signaled
106192025-04-30T06:13:28.907ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
106202025-04-30T06:13:28.907ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
106212025-04-30T06:13:28.907ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
106222025-04-30T06:13:28.907ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
106232025-04-30T06:13:28.908ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
106242025-04-30T06:13:28.910ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106252025-04-30T06:13:28.913ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106262025-04-30T06:13:28.913ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
106272025-04-30T06:13:28.914ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
106282025-04-30T06:13:28.922ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
106292025-04-30T06:13:28.922ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
106302025-04-30T06:13:28.922ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
106312025-04-30T06:13:28.924ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
106322025-04-30T06:13:28.934ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
106332025-04-30T06:13:28.934ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
106342025-04-30T06:13:28.937ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: fb ac 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:51914 req_id = 044bd4a5-87da-4ae0-957c-5038401f97e6
106352025-04-30T06:13:28.937ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64428,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51914 req_id = 044bd4a5-87da-4ae0-957c-5038401f97e6
106362025-04-30T06:13:28.937ZDEBGomicron-dev (store): query key key = _mgs._tcp
106372025-04-30T06:13:28.938ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
106382025-04-30T06:13:28.938ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51914 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 044bd4a5-87da-4ae0-957c-5038401f97e6
106392025-04-30T06:13:28.939ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2473, tv_nsec: 738210061 } })
106402025-04-30T06:13:28.939ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
106412025-04-30T06:13:28.940ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 72 bf 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e r...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63556 req_id = eead2d33-5531-480a-a428-16bc0de3d0f2
106422025-04-30T06:13:28.940ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29375,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63556 req_id = eead2d33-5531-480a-a428-16bc0de3d0f2
106432025-04-30T06:13:28.940ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
106442025-04-30T06:13:28.940ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63556 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = eead2d33-5531-480a-a428-16bc0de3d0f2
106452025-04-30T06:13:28.968ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
106462025-04-30T06:13:28.990ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
106472025-04-30T06:13:28.991ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: e0 93 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54971 req_id = e31488fd-3254-415f-ac2d-fc74ae10bca1
106482025-04-30T06:13:28.992ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57491,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54971 req_id = e31488fd-3254-415f-ac2d-fc74ae10bca1
106492025-04-30T06:13:28.993ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
106502025-04-30T06:13:28.995ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57491, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54971 req_id = e31488fd-3254-415f-ac2d-fc74ae10bca1
106512025-04-30T06:13:28.995ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
106522025-04-30T06:13:28.995ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
106532025-04-30T06:13:28.995ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
106542025-04-30T06:13:28.998ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "561f1cc8-3acc-4a7f-91e8-383d9ba07c73", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106552025-04-30T06:13:28.998ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
106562025-04-30T06:13:29.003ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "65cc3e40-9247-4465-a716-00dd15719c98", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106572025-04-30T06:13:29.005ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
106582025-04-30T06:13:29.010ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3558de6-5711-49e8-ac58-c380d805b8c5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106592025-04-30T06:13:29.010ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
106602025-04-30T06:13:29.012ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4569f5f-3978-4607-995b-b36638e85109", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106612025-04-30T06:13:29.012ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
106622025-04-30T06:13:29.014ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "72a06305-e132-4f1e-972a-bd4f2a098dbd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106632025-04-30T06:13:29.014ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
106642025-04-30T06:13:29.017ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b90f9fbf-b007-4c92-91a7-34b3c6a79c68", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106652025-04-30T06:13:29.017ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
106662025-04-30T06:13:29.019ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
106672025-04-30T06:13:29.019ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
106682025-04-30T06:13:29.019ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106692025-04-30T06:13:29.019ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
106702025-04-30T06:13:29.020ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9cd64051-03ac-40f4-870d-ff717adf481d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106712025-04-30T06:13:29.020ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
106722025-04-30T06:13:29.020ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
106732025-04-30T06:13:29.023ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ad1cb73-31ab-444b-b9f6-f3fa82ce3aa9", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106742025-04-30T06:13:29.023ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
106752025-04-30T06:13:29.024ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
106762025-04-30T06:13:29.024ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "324ea1b8-da9c-460d-a4a1-6fa708139fca", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106772025-04-30T06:13:29.024ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
106782025-04-30T06:13:29.028ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4567146-2fd1-44e9-8a28-b05a56d26e19", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106792025-04-30T06:13:29.029ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
106802025-04-30T06:13:29.031ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "971385c6-cdb8-4036-a72e-9e9f12fd9d01", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106812025-04-30T06:13:29.031ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
106822025-04-30T06:13:29.033ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4659d6e1-30d4-495e-824f-25a8daf3f6e9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106832025-04-30T06:13:29.034ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
106842025-04-30T06:13:29.035ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
106852025-04-30T06:13:29.035ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106862025-04-30T06:13:29.035ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
106872025-04-30T06:13:29.036ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
106882025-04-30T06:13:29.040ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
106892025-04-30T06:13:29.040ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
106902025-04-30T06:13:29.040ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
106912025-04-30T06:13:29.042ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc83f7f4-b795-447e-a259-36f341c809a5", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106922025-04-30T06:13:29.042ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
106932025-04-30T06:13:29.046ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e9c5761-c071-4030-9482-38ff4a02e6b9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106942025-04-30T06:13:29.046ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
106952025-04-30T06:13:29.049ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5875f96b-99c7-45cd-b3c9-8011981689cd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106962025-04-30T06:13:29.049ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
106972025-04-30T06:13:29.051ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ccf13f1b-d23a-4bd4-b573-e2eb9c061bb5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
106982025-04-30T06:13:29.051ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
106992025-04-30T06:13:29.053ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "66c16031-d411-49b0-8822-83ee76d58e88", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107002025-04-30T06:13:29.053ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
107012025-04-30T06:13:29.055ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e285ea33-eba5-4ed4-9367-892ff6c10b71", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107022025-04-30T06:13:29.055ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
107032025-04-30T06:13:29.057ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "669015b5-a5e8-4579-b1ac-c41a434f7318", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107042025-04-30T06:13:29.057ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
107052025-04-30T06:13:29.058ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "13105fb4-5386-4e61-b476-aedc6807acd9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107062025-04-30T06:13:29.058ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
107072025-04-30T06:13:29.059ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0170a2ce-4a04-418e-a038-d53cd2503153", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107082025-04-30T06:13:29.059ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
107092025-04-30T06:13:29.061ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "48786535-f1e8-4d17-8392-10a89559597c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107102025-04-30T06:13:29.062ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
107112025-04-30T06:13:29.064ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfff8904-cb1c-4dbd-9cb1-45a7535ed152", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107122025-04-30T06:13:29.064ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
107132025-04-30T06:13:29.065ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcb9d232-c602-45bb-ab32-ecfd1782791c", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107142025-04-30T06:13:29.065ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
107152025-04-30T06:13:29.067ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03ab2994-6d09-45fe-acbd-ae6129970feb", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107162025-04-30T06:13:29.067ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
107172025-04-30T06:13:29.070ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "16180c6f-6cde-47d4-ac2e-0924bddfe56b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107182025-04-30T06:13:29.070ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
107192025-04-30T06:13:29.071ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfefa72d-f5df-41f3-a86a-192ee0010ea2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107202025-04-30T06:13:29.071ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
107212025-04-30T06:13:29.073ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
107222025-04-30T06:13:29.073ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "000984bc-6b2f-4f63-bf6f-7c312abad98d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107232025-04-30T06:13:29.073ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
107242025-04-30T06:13:29.075ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e04b3bc-2e2a-4b70-bc5b-15982398c658", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107252025-04-30T06:13:29.076ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
107262025-04-30T06:13:29.078ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f498efa-5be7-45a6-a011-bf6e47f04849", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107272025-04-30T06:13:29.079ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
107282025-04-30T06:13:29.080ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43be6af2-b2ce-49d1-8133-2ba35d15c001", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107292025-04-30T06:13:29.080ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
107302025-04-30T06:13:29.082ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3353ca08-421c-40af-97e9-6a845b476284", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107312025-04-30T06:13:29.082ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
107322025-04-30T06:13:29.083ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e99bdde4-4f79-4125-87f7-4fc36aef01c1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107332025-04-30T06:13:29.083ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
107342025-04-30T06:13:29.084ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f02bb237-0d5c-430d-942c-9d9dbfcf1ff7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107352025-04-30T06:13:29.084ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
107362025-04-30T06:13:29.086ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1392c06f-dc2c-4b2e-b747-0eef41c7a549", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107372025-04-30T06:13:29.086ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
107382025-04-30T06:13:29.088ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "074512bd-76f2-46b9-9502-dc996c332b92", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107392025-04-30T06:13:29.088ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
107402025-04-30T06:13:29.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
107412025-04-30T06:13:29.089ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
107422025-04-30T06:13:29.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
107432025-04-30T06:13:29.090ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
107442025-04-30T06:13:29.092ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e82f68c-017e-4f09-8c13-1568ea6fe08b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107452025-04-30T06:13:29.092ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
107462025-04-30T06:13:29.094ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
107472025-04-30T06:13:29.094ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
107482025-04-30T06:13:29.095ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "11a1b67e-8d4c-40e2-a9f9-39e80cbcbc16", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107492025-04-30T06:13:29.096ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
107502025-04-30T06:13:29.097ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "37065592-7b14-44f6-bf8d-5f4d42aa68a4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107512025-04-30T06:13:29.097ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
107522025-04-30T06:13:29.102ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
107532025-04-30T06:13:29.102ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
107542025-04-30T06:13:29.102ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25a02f6f-28d3-404f-ad5b-9a32dbb5ed53", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107552025-04-30T06:13:29.102ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
107562025-04-30T06:13:29.104ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0fe4682-89e0-4f82-8a02-38d55be3a6c7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107572025-04-30T06:13:29.105ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
107582025-04-30T06:13:29.105ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "df424591-5343-4540-b3de-7c007dacf4ed", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107592025-04-30T06:13:29.106ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
107602025-04-30T06:13:29.107ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d8c8f51-73c1-4519-a25c-8fced54ca04b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107612025-04-30T06:13:29.107ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
107622025-04-30T06:13:29.109ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9ce2958-4bef-47ee-9051-e9ed4aa04f78", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107632025-04-30T06:13:29.109ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
107642025-04-30T06:13:29.111ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "91c0f79e-73c2-4a17-8f4d-de6cd1ee9116", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107652025-04-30T06:13:29.111ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
107662025-04-30T06:13:29.111ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107672025-04-30T06:13:29.111ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107682025-04-30T06:13:29.111ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107692025-04-30T06:13:29.112ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107702025-04-30T06:13:29.116ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107712025-04-30T06:13:29.121ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
107722025-04-30T06:13:29.123ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
107732025-04-30T06:13:29.123ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107742025-04-30T06:13:29.123ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107752025-04-30T06:13:29.123ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107762025-04-30T06:13:29.124ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
107772025-04-30T06:13:29.124ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107782025-04-30T06:13:29.124ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
107792025-04-30T06:13:29.124ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107802025-04-30T06:13:29.125ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
107812025-04-30T06:13:29.126ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107822025-04-30T06:13:29.127ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107832025-04-30T06:13:29.139ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
107842025-04-30T06:13:29.139ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
107852025-04-30T06:13:29.139ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
107862025-04-30T06:13:29.140ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
107872025-04-30T06:13:29.146ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
107882025-04-30T06:13:29.163ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
107892025-04-30T06:13:29.163ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
107902025-04-30T06:13:29.163ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
107912025-04-30T06:13:29.163ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
107922025-04-30T06:13:29.191ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
107932025-04-30T06:13:29.245ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
107942025-04-30T06:13:29.245ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
107952025-04-30T06:13:29.246ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:56848 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
107962025-04-30T06:13:29.248ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:56848 req_id = e99f7ec5-71ae-4297-86dc-510d8354953c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
107972025-04-30T06:13:29.249ZINFOomicron-dev (dropshot): request completed latency_us = 626 local_addr = [::1]:61881 method = GET remote_addr = [::1]:56848 req_id = e99f7ec5-71ae-4297-86dc-510d8354953c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
107982025-04-30T06:13:29.249ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e99f7ec5-71ae-4297-86dc-510d8354953c", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
107992025-04-30T06:13:29.251ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108002025-04-30T06:13:29.251ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
108012025-04-30T06:13:29.252ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108022025-04-30T06:13:29.254ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
108032025-04-30T06:13:29.254ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
108042025-04-30T06:13:29.254ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
108052025-04-30T06:13:29.256ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
108062025-04-30T06:13:29.256ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
108072025-04-30T06:13:29.256ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
108082025-04-30T06:13:29.256ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108092025-04-30T06:13:29.257ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108102025-04-30T06:13:29.261ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108112025-04-30T06:13:29.283ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
108122025-04-30T06:13:29.283ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
108132025-04-30T06:13:29.283ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108142025-04-30T06:13:29.284ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108152025-04-30T06:13:29.286ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
108162025-04-30T06:13:29.320ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
108172025-04-30T06:13:29.321ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
108182025-04-30T06:13:29.321ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 413.414461ms iteration = 38 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
108192025-04-30T06:13:29.321ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 39 reason = Signaled
108202025-04-30T06:13:29.321ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
108212025-04-30T06:13:29.321ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
108222025-04-30T06:13:29.321ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
108232025-04-30T06:13:29.321ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108242025-04-30T06:13:29.323ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108252025-04-30T06:13:29.326ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108262025-04-30T06:13:29.330ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
108272025-04-30T06:13:29.331ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
108282025-04-30T06:13:29.331ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
108292025-04-30T06:13:29.331ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
108302025-04-30T06:13:29.335ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
108312025-04-30T06:13:29.335ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
108322025-04-30T06:13:29.335ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 73 8a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 s............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61304 req_id = 12e3a812-4291-4c85-bd91-c95820426d12
108332025-04-30T06:13:29.335ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29578,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61304 req_id = 12e3a812-4291-4c85-bd91-c95820426d12
108342025-04-30T06:13:29.336ZDEBGomicron-dev (store): query key key = _mgs._tcp
108352025-04-30T06:13:29.336ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
108362025-04-30T06:13:29.336ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61304 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 12e3a812-4291-4c85-bd91-c95820426d12
108372025-04-30T06:13:29.336ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2474, tv_nsec: 135674613 } })
108382025-04-30T06:13:29.336ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
108392025-04-30T06:13:29.337ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
108402025-04-30T06:13:29.337ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
108412025-04-30T06:13:29.337ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108422025-04-30T06:13:29.337ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 7e 3c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ~<..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53245 req_id = a9bac1da-0421-4e9e-9544-14efddc032d3
108432025-04-30T06:13:29.337ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
108442025-04-30T06:13:29.337ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 32316,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53245 req_id = a9bac1da-0421-4e9e-9544-14efddc032d3
108452025-04-30T06:13:29.337ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
108462025-04-30T06:13:29.337ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53245 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a9bac1da-0421-4e9e-9544-14efddc032d3
108472025-04-30T06:13:29.337ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108482025-04-30T06:13:29.341ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108492025-04-30T06:13:29.347ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
108502025-04-30T06:13:29.348ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108512025-04-30T06:13:29.348ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
108522025-04-30T06:13:29.348ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108532025-04-30T06:13:29.351ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108542025-04-30T06:13:29.351ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
108552025-04-30T06:13:29.352ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108562025-04-30T06:13:29.378ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
108572025-04-30T06:13:29.430ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
108582025-04-30T06:13:29.430ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
108592025-04-30T06:13:29.431ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108602025-04-30T06:13:29.431ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
108612025-04-30T06:13:29.431ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108622025-04-30T06:13:29.435ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
108632025-04-30T06:13:29.438ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
108642025-04-30T06:13:29.444ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: e2 01 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50583 req_id = 1aa0ef39-9f09-4571-92cf-a0ddfbf897a8
108652025-04-30T06:13:29.444ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57857,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50583 req_id = 1aa0ef39-9f09-4571-92cf-a0ddfbf897a8
108662025-04-30T06:13:29.445ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
108672025-04-30T06:13:29.445ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57857, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50583 req_id = 1aa0ef39-9f09-4571-92cf-a0ddfbf897a8
108682025-04-30T06:13:29.445ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
108692025-04-30T06:13:29.446ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
108702025-04-30T06:13:29.446ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
108712025-04-30T06:13:29.452ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
108722025-04-30T06:13:29.452ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108732025-04-30T06:13:29.452ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
108742025-04-30T06:13:29.452ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bb2a77b-683c-49c6-b04e-cda5f4b3a1f2", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108752025-04-30T06:13:29.453ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
108762025-04-30T06:13:29.454ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108772025-04-30T06:13:29.464ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f45ffd9c-6fc6-4629-bf2d-4847896b522c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108782025-04-30T06:13:29.466ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
108792025-04-30T06:13:29.467ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
108802025-04-30T06:13:29.467ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
108812025-04-30T06:13:29.468ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
108822025-04-30T06:13:29.477ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7b91212-4f13-4575-8771-d0c2aee7c6ee", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108832025-04-30T06:13:29.477ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
108842025-04-30T06:13:29.479ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "354e5821-f37e-4192-afcc-b186b5aacb8c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108852025-04-30T06:13:29.479ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
108862025-04-30T06:13:29.482ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "57c5388a-448c-45dc-b637-ad47ecb6ed25", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108872025-04-30T06:13:29.482ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
108882025-04-30T06:13:29.484ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "71ad7b3c-f19a-48f5-ba42-7dc24faa0ff8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108892025-04-30T06:13:29.484ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
108902025-04-30T06:13:29.486ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "71f8a672-c562-45a7-bd71-11ccddd2e87e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108912025-04-30T06:13:29.486ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
108922025-04-30T06:13:29.489ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e1a9f83-ead1-450d-bf22-3c0b855d16b6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108932025-04-30T06:13:29.489ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
108942025-04-30T06:13:29.491ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce08468d-24ba-42a5-b4fc-ec428efd4249", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108952025-04-30T06:13:29.491ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
108962025-04-30T06:13:29.493ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2276063c-3db7-4858-8ed0-b0f245e64575", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108972025-04-30T06:13:29.494ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
108982025-04-30T06:13:29.501ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e86fbdeb-d15f-4a56-bb17-d374f1722de9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
108992025-04-30T06:13:29.502ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
109002025-04-30T06:13:29.505ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2914f96-ddf6-45bd-a066-b0bae710c1b9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109012025-04-30T06:13:29.506ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
109022025-04-30T06:13:29.509ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4280d51-c528-4779-b754-8ed6710ae677", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109032025-04-30T06:13:29.509ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
109042025-04-30T06:13:29.512ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "acc150a9-837d-4b1f-88d2-151a32b5d004", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109052025-04-30T06:13:29.512ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
109062025-04-30T06:13:29.516ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c47cb218-b60b-412a-bb1c-26bcb3c53d02", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109072025-04-30T06:13:29.516ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
109082025-04-30T06:13:29.520ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7176454c-a98a-4d44-8970-317a72b86296", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109092025-04-30T06:13:29.520ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
109102025-04-30T06:13:29.525ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "06058895-42bf-4f45-9378-15f9b0a6806e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109112025-04-30T06:13:29.525ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
109122025-04-30T06:13:29.535ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
109132025-04-30T06:13:29.538ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "464db86c-7d46-4410-849a-c7cc6233d268", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109142025-04-30T06:13:29.539ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
109152025-04-30T06:13:29.548ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5589fdee-ceab-4bb5-a1dd-ed6caa216843", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109162025-04-30T06:13:29.548ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
109172025-04-30T06:13:29.555ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "78a4fc96-7e08-4caf-b4b8-63c4fb86adb6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109182025-04-30T06:13:29.555ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
109192025-04-30T06:13:29.559ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c66b2394-20e7-4760-a544-8dfc84ea97d8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109202025-04-30T06:13:29.559ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
109212025-04-30T06:13:29.562ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de8406d0-30a1-42b5-9326-e1adf5461307", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109222025-04-30T06:13:29.562ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
109232025-04-30T06:13:29.564ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "364b908c-1afe-431a-870b-31293fec2337", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109242025-04-30T06:13:29.564ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
109252025-04-30T06:13:29.565ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "159ead4c-5fe5-42c2-a866-e25e14badbe2", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109262025-04-30T06:13:29.565ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
109272025-04-30T06:13:29.567ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1eeba358-74c7-4133-9e8f-653ba55902ff", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109282025-04-30T06:13:29.567ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
109292025-04-30T06:13:29.569ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "08724e4f-acd9-4142-9108-445061b549ce", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109302025-04-30T06:13:29.569ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
109312025-04-30T06:13:29.571ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9650c416-8975-43a9-a14a-3b8a28bc305d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109322025-04-30T06:13:29.571ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
109332025-04-30T06:13:29.573ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "481c2322-4018-45b9-9068-689922338b98", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109342025-04-30T06:13:29.573ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
109352025-04-30T06:13:29.575ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab256490-ae44-4199-b505-9e0db6815d1e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109362025-04-30T06:13:29.575ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
109372025-04-30T06:13:29.577ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7f186b8-4eca-40f5-984b-8b7afe23f53f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109382025-04-30T06:13:29.577ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
109392025-04-30T06:13:29.578ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "416a1618-e39b-45b8-8144-f1c5340d8356", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109402025-04-30T06:13:29.578ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
109412025-04-30T06:13:29.580ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7f34e81-cbab-408f-aed4-5a2bd70b618f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109422025-04-30T06:13:29.580ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
109432025-04-30T06:13:29.583ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "19572bc6-239b-4086-9397-27eb1e4f15ec", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109442025-04-30T06:13:29.583ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
109452025-04-30T06:13:29.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "665eddea-4592-4531-9cd0-b574ca5bd955", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109462025-04-30T06:13:29.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
109472025-04-30T06:13:29.586ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "087b64fb-2752-48a7-85df-d6ed4b51881b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109482025-04-30T06:13:29.586ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
109492025-04-30T06:13:29.586ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109502025-04-30T06:13:29.586ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109512025-04-30T06:13:29.586ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
109522025-04-30T06:13:29.586ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
109532025-04-30T06:13:29.588ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109542025-04-30T06:13:29.588ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a99c0f4-a4d3-4a50-9244-bb8c70f3c8fb", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
109552025-04-30T06:13:29.588ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
109562025-04-30T06:13:29.591ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109572025-04-30T06:13:29.596ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
109582025-04-30T06:13:29.596ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109592025-04-30T06:13:29.596ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
109602025-04-30T06:13:29.597ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109612025-04-30T06:13:29.600ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2132aeeb-227f-4fa6-adb9-c43148a7a792", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109622025-04-30T06:13:29.600ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
109632025-04-30T06:13:29.604ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
109642025-04-30T06:13:29.604ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
109652025-04-30T06:13:29.606ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
109662025-04-30T06:13:29.609ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b97ef622-5ec0-4992-9065-0d1621b82bea", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109672025-04-30T06:13:29.609ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
109682025-04-30T06:13:29.616ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0444459a-5c58-4876-9806-251eeb500219", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109692025-04-30T06:13:29.616ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
109702025-04-30T06:13:29.618ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ac30345-0be8-4949-85c2-43e24caaf01a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109712025-04-30T06:13:29.619ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
109722025-04-30T06:13:29.620ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68cd0d0d-908c-492e-914d-445090723e4f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109732025-04-30T06:13:29.621ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
109742025-04-30T06:13:29.621ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b490160-9cdf-4977-a5ed-6d1efabe4af5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109752025-04-30T06:13:29.622ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
109762025-04-30T06:13:29.623ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc0d72c6-f255-452f-8fdb-fd5b4dfde34f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109772025-04-30T06:13:29.623ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
109782025-04-30T06:13:29.624ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d71e15ed-94af-46aa-a137-0094ce88cc91", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109792025-04-30T06:13:29.624ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
109802025-04-30T06:13:29.625ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "41fb5e6d-86d8-4e5e-a2fe-8d76f2076197", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
109812025-04-30T06:13:29.625ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
109822025-04-30T06:13:29.625ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109832025-04-30T06:13:29.625ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109842025-04-30T06:13:29.625ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109852025-04-30T06:13:29.626ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109862025-04-30T06:13:29.629ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109872025-04-30T06:13:29.637ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
109882025-04-30T06:13:29.637ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
109892025-04-30T06:13:29.638ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
109902025-04-30T06:13:29.639ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109912025-04-30T06:13:29.639ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109922025-04-30T06:13:29.640ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109932025-04-30T06:13:29.643ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
109942025-04-30T06:13:29.647ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
109952025-04-30T06:13:29.647ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
109962025-04-30T06:13:29.647ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
109972025-04-30T06:13:29.648ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
109982025-04-30T06:13:29.648ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
109992025-04-30T06:13:29.701ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
110002025-04-30T06:13:29.701ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
110012025-04-30T06:13:29.701ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110022025-04-30T06:13:29.701ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
110032025-04-30T06:13:29.703ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110042025-04-30T06:13:29.727ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110052025-04-30T06:13:29.729ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110062025-04-30T06:13:29.733ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110072025-04-30T06:13:29.735ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110082025-04-30T06:13:29.736ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110092025-04-30T06:13:29.736ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
110102025-04-30T06:13:29.745ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
110112025-04-30T06:13:29.746ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
110122025-04-30T06:13:29.747ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:51439 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
110132025-04-30T06:13:29.748ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:51439 req_id = 6a0c224e-ded0-400e-ae8d-bcddbf558ec6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
110142025-04-30T06:13:29.750ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110152025-04-30T06:13:29.751ZINFOomicron-dev (dropshot): request completed latency_us = 2629 local_addr = [::1]:61881 method = GET remote_addr = [::1]:51439 req_id = 6a0c224e-ded0-400e-ae8d-bcddbf558ec6 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
110162025-04-30T06:13:29.752ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a0c224e-ded0-400e-ae8d-bcddbf558ec6", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:28 GMT"} })
110172025-04-30T06:13:29.757ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
110182025-04-30T06:13:29.757ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
110192025-04-30T06:13:29.757ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
110202025-04-30T06:13:29.763ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
110212025-04-30T06:13:29.763ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110222025-04-30T06:13:29.763ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
110232025-04-30T06:13:29.765ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110242025-04-30T06:13:29.765ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
110252025-04-30T06:13:29.765ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
110262025-04-30T06:13:29.765ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110272025-04-30T06:13:29.765ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110282025-04-30T06:13:29.766ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110292025-04-30T06:13:29.773ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110302025-04-30T06:13:29.773ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
110312025-04-30T06:13:29.773ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110322025-04-30T06:13:29.774ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110332025-04-30T06:13:29.780ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
110342025-04-30T06:13:29.780ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110352025-04-30T06:13:29.780ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110362025-04-30T06:13:29.781ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110372025-04-30T06:13:29.827ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
110382025-04-30T06:13:29.830ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
110392025-04-30T06:13:29.831ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
110402025-04-30T06:13:29.831ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 510.322454ms iteration = 39 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
110412025-04-30T06:13:29.831ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 40 reason = Signaled
110422025-04-30T06:13:29.832ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
110432025-04-30T06:13:29.832ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
110442025-04-30T06:13:29.832ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110452025-04-30T06:13:29.832ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110462025-04-30T06:13:29.834ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110472025-04-30T06:13:29.837ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110482025-04-30T06:13:29.843ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
110492025-04-30T06:13:29.844ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
110502025-04-30T06:13:29.844ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
110512025-04-30T06:13:29.845ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
110522025-04-30T06:13:29.851ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
110532025-04-30T06:13:29.851ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
110542025-04-30T06:13:29.852ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 0b 36 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .6..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55210 req_id = a68bf34a-8432-4756-85c0-898f903b55ec
110552025-04-30T06:13:29.852ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2870,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55210 req_id = a68bf34a-8432-4756-85c0-898f903b55ec
110562025-04-30T06:13:29.852ZDEBGomicron-dev (store): query key key = _mgs._tcp
110572025-04-30T06:13:29.853ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110582025-04-30T06:13:29.853ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55210 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = a68bf34a-8432-4756-85c0-898f903b55ec
110592025-04-30T06:13:29.853ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2474, tv_nsec: 652389013 } })
110602025-04-30T06:13:29.853ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
110612025-04-30T06:13:29.854ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a3 14 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60280 req_id = 7bcec89a-b49c-4933-babc-888f44a183f9
110622025-04-30T06:13:29.854ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41748,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60280 req_id = 7bcec89a-b49c-4933-babc-888f44a183f9
110632025-04-30T06:13:29.855ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
110642025-04-30T06:13:29.856ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60280 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 7bcec89a-b49c-4933-babc-888f44a183f9
110652025-04-30T06:13:29.927ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
110662025-04-30T06:13:29.928ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 2c f3 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ,............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:53796 req_id = 700f6482-27cd-485c-b628-e639539cfecb
110672025-04-30T06:13:29.928ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11507,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53796 req_id = 700f6482-27cd-485c-b628-e639539cfecb
110682025-04-30T06:13:29.928ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
110692025-04-30T06:13:29.929ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
110702025-04-30T06:13:29.929ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110712025-04-30T06:13:29.929ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
110722025-04-30T06:13:29.929ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
110732025-04-30T06:13:29.929ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 11507, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:53796 req_id = 700f6482-27cd-485c-b628-e639539cfecb
110742025-04-30T06:13:29.929ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
110752025-04-30T06:13:29.929ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
110762025-04-30T06:13:29.929ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
110772025-04-30T06:13:29.929ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110782025-04-30T06:13:29.935ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a04e2d3-dc4d-4247-b5d9-4cb42fe975eb", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
110792025-04-30T06:13:29.936ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
110802025-04-30T06:13:29.937ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
110812025-04-30T06:13:29.939ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "751defbb-d5c8-458d-ac28-1e28e02cddcb", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
110822025-04-30T06:13:29.939ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
110832025-04-30T06:13:29.943ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
110842025-04-30T06:13:29.943ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110852025-04-30T06:13:29.943ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
110862025-04-30T06:13:29.944ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110872025-04-30T06:13:29.948ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eb8785f-0b5d-4a83-b3bc-f5631740d40c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
110882025-04-30T06:13:29.948ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
110892025-04-30T06:13:29.948ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
110902025-04-30T06:13:29.948ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
110912025-04-30T06:13:29.949ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
110922025-04-30T06:13:29.953ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0450357-c9cb-4d90-9527-69d0eb6472a8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
110932025-04-30T06:13:29.953ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
110942025-04-30T06:13:29.957ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d50549e-2c6d-4fb9-85b4-f5273cf174b6", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
110952025-04-30T06:13:29.957ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
110962025-04-30T06:13:29.985ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8990fe4-6697-4d09-8490-bb3cab5f913f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
110972025-04-30T06:13:29.985ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
110982025-04-30T06:13:29.989ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7000a24f-e46b-4352-8b64-435998ebd267", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
110992025-04-30T06:13:29.990ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
111002025-04-30T06:13:29.995ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbd0973d-3e2c-4f5e-9a8c-1207f32d2683", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111012025-04-30T06:13:29.995ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
111022025-04-30T06:13:30.001ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb02a562-1e50-40ce-91c5-b474d64150ba", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111032025-04-30T06:13:30.001ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
111042025-04-30T06:13:30.003ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "826d8faa-80bb-44eb-acd8-11e6f0d7f87f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111052025-04-30T06:13:30.003ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
111062025-04-30T06:13:30.005ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "620f6b74-cb40-467f-8282-9f3ad2ab9771", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111072025-04-30T06:13:30.005ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
111082025-04-30T06:13:30.007ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "84201218-a2c5-4eb3-b764-3c733ed19401", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111092025-04-30T06:13:30.007ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
111102025-04-30T06:13:30.009ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "95b67a89-73a7-4536-981c-603aefc443bc", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111112025-04-30T06:13:30.009ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
111122025-04-30T06:13:30.016ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac591e9b-f0d2-47a9-bf61-2e188729fe9d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111132025-04-30T06:13:30.016ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
111142025-04-30T06:13:30.019ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "54de030c-2a9a-446a-81e3-5a6356119947", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111152025-04-30T06:13:30.019ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
111162025-04-30T06:13:30.019ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
111172025-04-30T06:13:30.023ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "59b9efd4-489b-4ab9-ada3-d4f5cbced8b4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111182025-04-30T06:13:30.023ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
111192025-04-30T06:13:30.028ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "38aba6b2-67f3-4e16-a1e5-266f63f96a4f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111202025-04-30T06:13:30.028ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
111212025-04-30T06:13:30.032ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "33fbefe0-fa2a-46b4-90c8-5a8ea81dbf31", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111222025-04-30T06:13:30.032ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
111232025-04-30T06:13:30.037ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6e29393-f8aa-49f9-94ae-d2f3e02aedba", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111242025-04-30T06:13:30.037ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
111252025-04-30T06:13:30.038ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc70aef1-6be8-4051-87fe-db18ee829014", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111262025-04-30T06:13:30.039ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
111272025-04-30T06:13:30.042ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88d0a45e-f86d-4f9d-8ec1-87cff0fcfd1d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111282025-04-30T06:13:30.043ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
111292025-04-30T06:13:30.044ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31be02c4-de5f-452d-9b48-07f27473c797", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111302025-04-30T06:13:30.045ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
111312025-04-30T06:13:30.047ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "04928409-aab5-46b1-a280-b34d8b9f2378", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111322025-04-30T06:13:30.047ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
111332025-04-30T06:13:30.050ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "92f5a66b-0b27-418b-902c-0059c2488625", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111342025-04-30T06:13:30.051ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
111352025-04-30T06:13:30.053ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab8d54d2-49cf-4ac7-ad4f-079a22a0091b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111362025-04-30T06:13:30.053ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
111372025-04-30T06:13:30.055ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb2e6825-c452-4e00-b06b-61fe95445d66", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111382025-04-30T06:13:30.055ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
111392025-04-30T06:13:30.057ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1220e63a-ce6d-4870-a7a7-dea35c944e7b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111402025-04-30T06:13:30.057ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
111412025-04-30T06:13:30.059ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f476cf3-d4c8-4c05-8bdb-58d641fa2fe2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111422025-04-30T06:13:30.060ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
111432025-04-30T06:13:30.062ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "99a830c9-bc29-44d6-84fd-5bf879fd6ac1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111442025-04-30T06:13:30.062ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
111452025-04-30T06:13:30.065ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0650cd3d-f17a-405a-b708-cdcb09482c8e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111462025-04-30T06:13:30.065ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
111472025-04-30T06:13:30.065ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "13bcf4cd-e384-4ed0-a258-8a5c9150f55b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111482025-04-30T06:13:30.065ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
111492025-04-30T06:13:30.068ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb713f8b-bbb3-48e6-a40d-523e96257c9a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111502025-04-30T06:13:30.068ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
111512025-04-30T06:13:30.069ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b34c7631-9dde-410f-a837-c36b62ad0a31", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111522025-04-30T06:13:30.069ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
111532025-04-30T06:13:30.070ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31a2deae-851b-47b0-bff4-52fd01a3b3cb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111542025-04-30T06:13:30.070ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
111552025-04-30T06:13:30.071ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cf87400-bbdd-43fb-8e12-771a53ee2dda", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111562025-04-30T06:13:30.071ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
111572025-04-30T06:13:30.073ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc7902a1-a0ec-4e75-9e93-3cfa67c364b0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111582025-04-30T06:13:30.073ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
111592025-04-30T06:13:30.084ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
111602025-04-30T06:13:30.084ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
111612025-04-30T06:13:30.084ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111622025-04-30T06:13:30.084ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
111632025-04-30T06:13:30.086ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111642025-04-30T06:13:30.089ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
111652025-04-30T06:13:30.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
111662025-04-30T06:13:30.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
111672025-04-30T06:13:30.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
111682025-04-30T06:13:30.091ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7df9878-c356-451e-aac8-8a5cd7e92b50", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111692025-04-30T06:13:30.091ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
111702025-04-30T06:13:30.093ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "91a472bd-d225-461f-a2eb-eb1fd24fc459", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111712025-04-30T06:13:30.093ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
111722025-04-30T06:13:30.095ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
111732025-04-30T06:13:30.096ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "91d6f8bf-23db-42dd-9c60-4446d29b6f1d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111742025-04-30T06:13:30.097ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
111752025-04-30T06:13:30.097ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
111762025-04-30T06:13:30.097ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
111772025-04-30T06:13:30.097ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
111782025-04-30T06:13:30.097ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
111792025-04-30T06:13:30.100ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
111802025-04-30T06:13:30.100ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
111812025-04-30T06:13:30.103ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
111822025-04-30T06:13:30.103ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
111832025-04-30T06:13:30.105ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef08b56c-7fb2-42de-88b8-b6e6bd02a6c8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111842025-04-30T06:13:30.106ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
111852025-04-30T06:13:30.109ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f4c8dad-4ff6-4c73-a8cb-ae9ae8673f0d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111862025-04-30T06:13:30.109ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
111872025-04-30T06:13:30.110ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "21a3cde9-be50-4080-b888-e7de970755ca", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
111882025-04-30T06:13:30.111ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
111892025-04-30T06:13:30.113ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1dad3c2-e42c-4544-a152-bc33da24a6fe", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111902025-04-30T06:13:30.113ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
111912025-04-30T06:13:30.115ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f5bbefa-bee5-443b-bd72-00cc49bb4e82", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111922025-04-30T06:13:30.115ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
111932025-04-30T06:13:30.116ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fe8d566-3b29-4494-934b-ab663b7db16e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
111942025-04-30T06:13:30.117ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
111952025-04-30T06:13:30.117ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
111962025-04-30T06:13:30.117ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
111972025-04-30T06:13:30.117ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
111982025-04-30T06:13:30.118ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
111992025-04-30T06:13:30.121ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112002025-04-30T06:13:30.121ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
112012025-04-30T06:13:30.123ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112022025-04-30T06:13:30.126ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112032025-04-30T06:13:30.130ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
112042025-04-30T06:13:30.131ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
112052025-04-30T06:13:30.131ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112062025-04-30T06:13:30.131ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112072025-04-30T06:13:30.131ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112082025-04-30T06:13:30.131ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112092025-04-30T06:13:30.133ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112102025-04-30T06:13:30.137ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
112112025-04-30T06:13:30.137ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112122025-04-30T06:13:30.137ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112132025-04-30T06:13:30.137ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112142025-04-30T06:13:30.159ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
112152025-04-30T06:13:30.184ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
112162025-04-30T06:13:30.185ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
112172025-04-30T06:13:30.185ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:61446 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
112182025-04-30T06:13:30.186ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:61446 req_id = 65e63f5b-e17f-48e9-8778-14e8cf01e141 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
112192025-04-30T06:13:30.187ZINFOomicron-dev (dropshot): request completed latency_us = 551 local_addr = [::1]:61881 method = GET remote_addr = [::1]:61446 req_id = 65e63f5b-e17f-48e9-8778-14e8cf01e141 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
112202025-04-30T06:13:30.188ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "65e63f5b-e17f-48e9-8778-14e8cf01e141", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
112212025-04-30T06:13:30.191ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
112222025-04-30T06:13:30.194ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
112232025-04-30T06:13:30.194ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
112242025-04-30T06:13:30.194ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
112252025-04-30T06:13:30.202ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
112262025-04-30T06:13:30.203ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112272025-04-30T06:13:30.203ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112282025-04-30T06:13:30.203ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112292025-04-30T06:13:30.204ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112302025-04-30T06:13:30.210ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
112312025-04-30T06:13:30.210ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112322025-04-30T06:13:30.211ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112332025-04-30T06:13:30.211ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112342025-04-30T06:13:30.211ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
112352025-04-30T06:13:30.212ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112362025-04-30T06:13:30.218ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
112372025-04-30T06:13:30.219ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112382025-04-30T06:13:30.219ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112392025-04-30T06:13:30.219ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112402025-04-30T06:13:30.222ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112412025-04-30T06:13:30.244ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
112422025-04-30T06:13:30.244ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112432025-04-30T06:13:30.244ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
112442025-04-30T06:13:30.246ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112452025-04-30T06:13:30.267ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
112462025-04-30T06:13:30.267ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
112472025-04-30T06:13:30.268ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 436.221944ms iteration = 40 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
112482025-04-30T06:13:30.269ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 41 reason = Signaled
112492025-04-30T06:13:30.269ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
112502025-04-30T06:13:30.269ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112512025-04-30T06:13:30.269ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112522025-04-30T06:13:30.270ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112532025-04-30T06:13:30.275ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112542025-04-30T06:13:30.276ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
112552025-04-30T06:13:30.276ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
112562025-04-30T06:13:30.277ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
112572025-04-30T06:13:30.280ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
112582025-04-30T06:13:30.285ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
112592025-04-30T06:13:30.285ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
112602025-04-30T06:13:30.285ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
112612025-04-30T06:13:30.286ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
112622025-04-30T06:13:30.289ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
112632025-04-30T06:13:30.289ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
112642025-04-30T06:13:30.291ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 5b d3 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 [............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61139 req_id = 5cdc82ec-3416-4c8d-b3b1-053c863092b9
112652025-04-30T06:13:30.291ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23507,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61139 req_id = 5cdc82ec-3416-4c8d-b3b1-053c863092b9
112662025-04-30T06:13:30.292ZDEBGomicron-dev (store): query key key = _mgs._tcp
112672025-04-30T06:13:30.292ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
112682025-04-30T06:13:30.292ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61139 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5cdc82ec-3416-4c8d-b3b1-053c863092b9
112692025-04-30T06:13:30.293ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2475, tv_nsec: 92092944 } })
112702025-04-30T06:13:30.293ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
112712025-04-30T06:13:30.294ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 50 0c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e P...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62730 req_id = 567934d4-aff3-4f0e-a52a-3de9b339a421
112722025-04-30T06:13:30.294ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20492,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62730 req_id = 567934d4-aff3-4f0e-a52a-3de9b339a421
112732025-04-30T06:13:30.294ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
112742025-04-30T06:13:30.294ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62730 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 567934d4-aff3-4f0e-a52a-3de9b339a421
112752025-04-30T06:13:30.320ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
112762025-04-30T06:13:30.327ZDEBGomicron-dev (BackgroundTasks): activating background_task = nat_v4_garbage_collector iteration = 2 reason = Timeout
112772025-04-30T06:13:30.327ZDEBGomicron-dev (BackgroundTasks): activating background_task = phantom_disks iteration = 2 reason = Timeout
112782025-04-30T06:13:30.328ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database
112792025-04-30T06:13:30.328ZDEBGomicron-dev (BackgroundTasks): roles background_task = nat_v4_garbage_collector roles = RoleSet { roles: {} }
112802025-04-30T06:13:30.329ZDEBGomicron-dev (BackgroundTasks): activating background_task = bfd_manager iteration = 2 reason = Timeout
112812025-04-30T06:13:30.329ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database
112822025-04-30T06:13:30.329ZDEBGomicron-dev (BackgroundTasks): roles background_task = bfd_manager roles = RoleSet { roles: {} }
112832025-04-30T06:13:30.329ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = nat_v4_garbage_collector resource = Database result = Ok(())
112842025-04-30T06:13:30.329ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = bfd_manager resource = Database result = Ok(())
112852025-04-30T06:13:30.330ZDEBGomicron-dev (BackgroundTasks): activating background_task = support_bundle_collector iteration = 2 reason = Timeout
112862025-04-30T06:13:30.330ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
112872025-04-30T06:13:30.330ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112882025-04-30T06:13:30.330ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
112892025-04-30T06:13:30.330ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
112902025-04-30T06:13:30.330ZDEBGomicron-dev (BackgroundTasks): activating background_task = physical_disk_adoption iteration = 3 reason = Timeout
112912025-04-30T06:13:30.330ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet
112922025-04-30T06:13:30.330ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = physical_disk_adoption resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
112932025-04-30T06:13:30.330ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
112942025-04-30T06:13:30.330ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
112952025-04-30T06:13:30.330ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
112962025-04-30T06:13:30.331ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
112972025-04-30T06:13:30.332ZDEBGomicron-dev (BackgroundTasks): activating background_task = v2p_manager iteration = 2 reason = Timeout
112982025-04-30T06:13:30.332ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
112992025-04-30T06:13:30.332ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
113002025-04-30T06:13:30.332ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
113012025-04-30T06:13:30.332ZDEBGomicron-dev (BackgroundTasks): activating background_task = read_only_region_replacement_start iteration = 2 reason = Timeout
113022025-04-30T06:13:30.332ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database
113032025-04-30T06:13:30.332ZDEBGomicron-dev (BackgroundTasks): roles background_task = read_only_region_replacement_start roles = RoleSet { roles: {} }
113042025-04-30T06:13:30.333ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = read_only_region_replacement_start resource = Database result = Ok(())
113052025-04-30T06:13:30.333ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_snapshot_replacement_garbage_collection iteration = 2 reason = Timeout
113062025-04-30T06:13:30.333ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database
113072025-04-30T06:13:30.333ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_garbage_collection roles = RoleSet { roles: {} }
113082025-04-30T06:13:30.334ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_garbage_collection resource = Database result = Ok(())
113092025-04-30T06:13:30.334ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_snapshot_replacement_finish iteration = 2 reason = Timeout
113102025-04-30T06:13:30.334ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database
113112025-04-30T06:13:30.334ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_finish roles = RoleSet { roles: {} }
113122025-04-30T06:13:30.334ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_finish resource = Database result = Ok(())
113132025-04-30T06:13:30.334ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_snapshot_replacement_start iteration = 2 reason = Timeout
113142025-04-30T06:13:30.334ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
113152025-04-30T06:13:30.334ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
113162025-04-30T06:13:30.335ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
113172025-04-30T06:13:30.335ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = phantom_disks elapsed = 7.786985ms iteration = 2 status = {"phantom_disk_deleted_err":0,"phantom_disk_deleted_ok":0}
113182025-04-30T06:13:30.335ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_snapshot_replacement_step iteration = 2 reason = Timeout
113192025-04-30T06:13:30.335ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
113202025-04-30T06:13:30.335ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
113212025-04-30T06:13:30.336ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
113222025-04-30T06:13:30.336ZDEBGomicron-dev (BackgroundTasks): activating background_task = vpc_route_manager iteration = 2 reason = Timeout
113232025-04-30T06:13:30.336ZINFOomicron-dev (BackgroundTasks): VPC route manager running background_task = vpc_route_manager
113242025-04-30T06:13:30.336ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
113252025-04-30T06:13:30.336ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
113262025-04-30T06:13:30.336ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
113272025-04-30T06:13:30.336ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
113282025-04-30T06:13:30.336ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
113292025-04-30T06:13:30.336ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
113302025-04-30T06:13:30.337ZDEBGomicron-dev (BackgroundTasks): activating background_task = switch_port_config_manager iteration = 2 reason = Timeout
113312025-04-30T06:13:30.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet
113322025-04-30T06:13:30.337ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = switch_port_config_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
113332025-04-30T06:13:30.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
113342025-04-30T06:13:30.337ZDEBGomicron-dev (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
113352025-04-30T06:13:30.337ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
113362025-04-30T06:13:30.337ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_replacement iteration = 2 reason = Timeout
113372025-04-30T06:13:30.337ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
113382025-04-30T06:13:30.337ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
113392025-04-30T06:13:30.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
113402025-04-30T06:13:30.338ZDEBGomicron-dev (BackgroundTasks): activating background_task = service_zone_nat_tracker iteration = 2 reason = Timeout
113412025-04-30T06:13:30.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory
113422025-04-30T06:13:30.338ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
113432025-04-30T06:13:30.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
113442025-04-30T06:13:30.338ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
113452025-04-30T06:13:30.338ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
113462025-04-30T06:13:30.338ZDEBGomicron-dev (BackgroundTasks): activating background_task = instance_watcher iteration = 2 reason = Timeout
113472025-04-30T06:13:30.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet
113482025-04-30T06:13:30.338ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = instance_watcher resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
113492025-04-30T06:13:30.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
113502025-04-30T06:13:30.339ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
113512025-04-30T06:13:30.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
113522025-04-30T06:13:30.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = region_replacement_driver iteration = 2 reason = Timeout
113532025-04-30T06:13:30.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = instance_updater iteration = 2 reason = Timeout
113542025-04-30T06:13:30.340ZDEBGomicron-dev (BackgroundTasks): looking for instances with destroyed active VMMs... background_task = instance_updater
113552025-04-30T06:13:30.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database
113562025-04-30T06:13:30.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
113572025-04-30T06:13:30.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
113582025-04-30T06:13:30.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
113592025-04-30T06:13:30.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = blueprint_loader iteration = 4 reason = Timeout
113602025-04-30T06:13:30.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
113612025-04-30T06:13:30.340ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
113622025-04-30T06:13:30.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
113632025-04-30T06:13:30.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
113642025-04-30T06:13:30.341ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 5f 1b 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 _............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:63207 req_id = ff41ae96-faa1-465b-b265-789c31f06414
113652025-04-30T06:13:30.341ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24347,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63207 req_id = ff41ae96-faa1-465b-b265-789c31f06414
113662025-04-30T06:13:30.341ZDEBGomicron-dev (store): query key key = _dendrite._tcp
113672025-04-30T06:13:30.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
113682025-04-30T06:13:30.341ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113692025-04-30T06:13:30.341ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63207 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = ff41ae96-faa1-465b-b265-789c31f06414
113702025-04-30T06:13:30.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
113712025-04-30T06:13:30.341ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2475, tv_nsec: 139773885 } })
113722025-04-30T06:13:30.341ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
113732025-04-30T06:13:30.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
113742025-04-30T06:13:30.341ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f9 11 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61669 req_id = d10696b2-4518-410a-99c9-509986ce36c7
113752025-04-30T06:13:30.341ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63761,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61669 req_id = d10696b2-4518-410a-99c9-509986ce36c7
113762025-04-30T06:13:30.341ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113772025-04-30T06:13:30.341ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61669 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d10696b2-4518-410a-99c9-509986ce36c7
113782025-04-30T06:13:30.342ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
113792025-04-30T06:13:30.342ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
113802025-04-30T06:13:30.343ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113812025-04-30T06:13:30.344ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
113822025-04-30T06:13:30.346ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 02 a2 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:52071 req_id = d7832a68-aae2-42ae-b618-6b13ecf812c2
113832025-04-30T06:13:30.346ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 674,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52071 req_id = d7832a68-aae2-42ae-b618-6b13ecf812c2
113842025-04-30T06:13:30.346ZDEBGomicron-dev (store): query key key = _dendrite._tcp
113852025-04-30T06:13:30.346ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113862025-04-30T06:13:30.346ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52071 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d7832a68-aae2-42ae-b618-6b13ecf812c2
113872025-04-30T06:13:30.346ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2475, tv_nsec: 144285995 } })
113882025-04-30T06:13:30.346ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
113892025-04-30T06:13:30.346ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d7 ce 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64094 req_id = ba1b87b7-8285-49a6-bd09-1444cea104cb
113902025-04-30T06:13:30.346ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55246,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64094 req_id = ba1b87b7-8285-49a6-bd09-1444cea104cb
113912025-04-30T06:13:30.346ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
113922025-04-30T06:13:30.347ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64094 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = ba1b87b7-8285-49a6-bd09-1444cea104cb
113932025-04-30T06:13:30.347ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
113942025-04-30T06:13:30.349ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
113952025-04-30T06:13:30.349ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
113962025-04-30T06:13:30.349ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
113972025-04-30T06:13:30.349ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
113982025-04-30T06:13:30.349ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
113992025-04-30T06:13:30.349ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
114002025-04-30T06:13:30.349ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114012025-04-30T06:13:30.349ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114022025-04-30T06:13:30.365ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
114032025-04-30T06:13:30.365ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
114042025-04-30T06:13:30.366ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
114052025-04-30T06:13:30.375ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_snapshot_replacement_garbage_collection elapsed = 41.652039ms iteration = 2 status = {"errors":[],"garbage_collect_requested":[]}
114062025-04-30T06:13:30.378ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
114072025-04-30T06:13:30.378ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114082025-04-30T06:13:30.380ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114092025-04-30T06:13:30.380ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
114102025-04-30T06:13:30.381ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database
114112025-04-30T06:13:30.381ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_start roles = RoleSet { roles: {} }
114122025-04-30T06:13:30.382ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_start resource = Database result = Ok(())
114132025-04-30T06:13:30.384ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114142025-04-30T06:13:30.384ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114152025-04-30T06:13:30.388ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
114162025-04-30T06:13:30.388ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet
114172025-04-30T06:13:30.388ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = vpc_route_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114182025-04-30T06:13:30.388ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
114192025-04-30T06:13:30.388ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
114202025-04-30T06:13:30.388ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
114212025-04-30T06:13:30.401ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114222025-04-30T06:13:30.407ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114232025-04-30T06:13:30.408ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Inventory result = Ok(())
114242025-04-30T06:13:30.408ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
114252025-04-30T06:13:30.408ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
114262025-04-30T06:13:30.408ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Fleet result = Ok(())
114272025-04-30T06:13:30.408ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database
114282025-04-30T06:13:30.408ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_watcher roles = RoleSet { roles: {} }
114292025-04-30T06:13:30.408ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
114302025-04-30T06:13:30.408ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_watcher resource = Database result = Ok(())
114312025-04-30T06:13:30.414ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114322025-04-30T06:13:30.419ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
114332025-04-30T06:13:30.419ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
114342025-04-30T06:13:30.419ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
114352025-04-30T06:13:30.420ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
114362025-04-30T06:13:30.428ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet
114372025-04-30T06:13:30.428ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = physical_disk_adoption resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
114382025-04-30T06:13:30.428ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
114392025-04-30T06:13:30.428ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
114402025-04-30T06:13:30.429ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
114412025-04-30T06:13:30.467ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
114422025-04-30T06:13:30.469ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: f2 f8 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63576 req_id = ed859ddc-2b40-451e-bc9c-6e2ffb3a863d
114432025-04-30T06:13:30.469ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 62200,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63576 req_id = ed859ddc-2b40-451e-bc9c-6e2ffb3a863d
114442025-04-30T06:13:30.469ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
114452025-04-30T06:13:30.472ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 62200, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63576 req_id = ed859ddc-2b40-451e-bc9c-6e2ffb3a863d
114462025-04-30T06:13:30.472ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
114472025-04-30T06:13:30.472ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
114482025-04-30T06:13:30.472ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
114492025-04-30T06:13:30.473ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "850c394b-2d4f-4c80-95b5-71dccc2b1d3c", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114502025-04-30T06:13:30.473ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
114512025-04-30T06:13:30.475ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad8b16b7-449d-4c88-9355-e48dab5e6f0c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114522025-04-30T06:13:30.475ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
114532025-04-30T06:13:30.478ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "510620e4-5c59-41a3-acf2-d39b6ee865de", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114542025-04-30T06:13:30.478ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
114552025-04-30T06:13:30.479ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e53c06f0-d880-4783-a3cc-6626875acfa7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114562025-04-30T06:13:30.480ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
114572025-04-30T06:13:30.481ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca305170-0995-4f78-b660-7709ce484beb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114582025-04-30T06:13:30.481ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
114592025-04-30T06:13:30.483ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c65bb34b-32ab-4ddc-a920-a005548b4cbb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114602025-04-30T06:13:30.483ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
114612025-04-30T06:13:30.484ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfe7e0d8-a611-4257-964e-d56921c0927f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114622025-04-30T06:13:30.485ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
114632025-04-30T06:13:30.487ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cf65992-664b-45bc-9043-9777b19e4a6a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114642025-04-30T06:13:30.487ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
114652025-04-30T06:13:30.487ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3abe3c72-2a33-4563-98d5-5fa96d3b4149", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114662025-04-30T06:13:30.487ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
114672025-04-30T06:13:30.491ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "450a433d-ec31-40bc-bff9-e7d829be73b4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114682025-04-30T06:13:30.491ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
114692025-04-30T06:13:30.496ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "69275491-2258-43e1-aaf3-7a2bb3fc8cfd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
114702025-04-30T06:13:30.496ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
114712025-04-30T06:13:30.498ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e6df1b5-21ba-48a5-a7db-b2d820b1b25c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114722025-04-30T06:13:30.499ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
114732025-04-30T06:13:30.499ZDEBGomicron-dev (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114742025-04-30T06:13:30.500ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "90010f77-9230-4b8d-8abc-247841449fbd", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114752025-04-30T06:13:30.500ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
114762025-04-30T06:13:30.502ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b48c40bc-cb72-4e2a-a765-acd295b23048", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114772025-04-30T06:13:30.502ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
114782025-04-30T06:13:30.506ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed7d0c9d-5814-4fca-bea3-12a7abc6881f", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114792025-04-30T06:13:30.506ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
114802025-04-30T06:13:30.509ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = read_only_region_replacement_start elapsed = 176.516428ms iteration = 2 status = {"errors":[],"requests_created_ok":[]}
114812025-04-30T06:13:30.512ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
114822025-04-30T06:13:30.517ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
114832025-04-30T06:13:30.517ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
114842025-04-30T06:13:30.517ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
114852025-04-30T06:13:30.530ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
114862025-04-30T06:13:30.530ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fde4343f-1c8c-461b-bef7-2d85f12fdae1", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
114872025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
114882025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfc09d75-c03d-4d3f-adf4-e0a4676372e2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
114892025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
114902025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "34494a23-fad3-46f0-8275-71564ebe1e2d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
114912025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
114922025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1d2a1d7-6194-47ce-a148-e810fb8ecd57", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
114932025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
114942025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "53eb47f4-e918-4232-b879-c1edc9e21941", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
114952025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
114962025-04-30T06:13:30.531ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4332ee17-ae61-4fc0-8288-ca5a3f7d535a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
114972025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
114982025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Fleet result = Ok(())
114992025-04-30T06:13:30.558ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database
115002025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3e61e46f-8e46-4b78-917b-7bb603204d96", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
115012025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): roles background_task = switch_port_config_manager roles = RoleSet { roles: {} }
115022025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
115032025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = switch_port_config_manager resource = Database result = Ok(())
115042025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "47292fd6-8e6f-4164-a9ac-38a74a8ca20e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115052025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
115062025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "89ad9d9f-f5e9-4496-8450-502688c33488", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115072025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
115082025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "60e90859-bfbb-49dd-b450-de16bee73434", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115092025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
115102025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "42b177ec-d20d-4add-9d1a-b970640b977d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
115112025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
115122025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4952124-b584-49f1-ac7e-4f4299c683a7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
115132025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
115142025-04-30T06:13:30.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c771011d-a618-41c1-949d-7581b1a4f98f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115152025-04-30T06:13:30.559ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
115162025-04-30T06:13:30.559ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "59000bf0-70f9-4e9a-a0fd-0e37c1bc05a2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115172025-04-30T06:13:30.559ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
115182025-04-30T06:13:30.561ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7449f7a2-192f-41f3-b2e6-e4c3d42637d1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115192025-04-30T06:13:30.561ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
115202025-04-30T06:13:30.570ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d9af72c-e873-49c2-a5be-dd362a0311cc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
115212025-04-30T06:13:30.570ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
115222025-04-30T06:13:30.570ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad657bdd-17c9-490c-b7d2-ace79275222b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115232025-04-30T06:13:30.570ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
115242025-04-30T06:13:30.570ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9395697-7745-4ba4-8cb8-9c2033a5673d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115252025-04-30T06:13:30.571ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
115262025-04-30T06:13:30.571ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "619a7ea1-bbd3-4834-9968-7a504476dfbe", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115272025-04-30T06:13:30.571ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
115282025-04-30T06:13:30.571ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "be25a64c-ba26-4e5a-bf37-9f1c64280436", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115292025-04-30T06:13:30.572ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
115302025-04-30T06:13:30.584ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e10658c-72ba-45b4-907a-61a913a1a5d2", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115312025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
115322025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a64367ab-b139-4708-bebd-3425b0eec3f3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115332025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
115342025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5db73dd1-6e16-40e9-9057-cd841aa4de45", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115352025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
115362025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "391ceffa-1eb9-4021-a1ad-1e4a7a1c9cae", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115372025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
115382025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "69b7346f-eb9e-4802-b5dd-226b11912cbb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115392025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
115402025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c75641df-a7fc-4188-b6bd-a218299fae3d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115412025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
115422025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9a5766b-1e36-40ac-94d5-6573f62e04a1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115432025-04-30T06:13:30.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
115442025-04-30T06:13:30.586ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ed7d68b-2632-4b52-a93b-664dfdb02d09", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:29 GMT"} })
115452025-04-30T06:13:30.586ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
115462025-04-30T06:13:30.588ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef0376ec-28aa-4b84-85d6-ea6bae692616", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115472025-04-30T06:13:30.588ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
115482025-04-30T06:13:30.592ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "41744657-fa09-499e-a139-c8aeb830fa5d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
115492025-04-30T06:13:30.592ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
115502025-04-30T06:13:30.592ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
115512025-04-30T06:13:30.592ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
115522025-04-30T06:13:30.592ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
115532025-04-30T06:13:30.592ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
115542025-04-30T06:13:30.610ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
115552025-04-30T06:13:30.610ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
115562025-04-30T06:13:30.611ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
115572025-04-30T06:13:30.615ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bfc4639-5ca6-4323-a678-d4a0347a6fdf, lookup_type: ById(1bfc4639-5ca6-4323-a678-d4a0347a6fdf) }
115582025-04-30T06:13:30.615ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
115592025-04-30T06:13:30.615ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
115602025-04-30T06:13:30.615ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
115612025-04-30T06:13:30.616ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
115622025-04-30T06:13:30.618ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database
115632025-04-30T06:13:30.618ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_replacement roles = RoleSet { roles: {} }
115642025-04-30T06:13:30.619ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement resource = Database result = Ok(())
115652025-04-30T06:13:30.622ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_snapshot_replacement_finish elapsed = 288.410419ms iteration = 2 status = {"errors":[],"finish_invoked_ok":[]}
115662025-04-30T06:13:30.624ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
115672025-04-30T06:13:30.624ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
115682025-04-30T06:13:30.625ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
115692025-04-30T06:13:30.628ZINFOomicron-dev (BackgroundTasks): listed instances with destroyed active VMMs background_task = instance_updater count = 0
115702025-04-30T06:13:30.628ZDEBGomicron-dev (BackgroundTasks): looking for instances with failed active VMMs... background_task = instance_updater
115712025-04-30T06:13:30.628ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database
115722025-04-30T06:13:30.628ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
115732025-04-30T06:13:30.628ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
115742025-04-30T06:13:30.634ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database
115752025-04-30T06:13:30.634ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_replacement_driver roles = RoleSet { roles: {} }
115762025-04-30T06:13:30.636ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_replacement_driver resource = Database result = Ok(())
115772025-04-30T06:13:30.667ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115782025-04-30T06:13:30.668ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
115792025-04-30T06:13:30.668ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
115802025-04-30T06:13:30.669ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
115812025-04-30T06:13:30.669ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
115822025-04-30T06:13:30.669ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: f7 70 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .p..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:50499 req_id = 1e7e9e35-69f5-4724-a2d7-04bcc6b96269
115832025-04-30T06:13:30.669ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 63344,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50499 req_id = 1e7e9e35-69f5-4724-a2d7-04bcc6b96269
115842025-04-30T06:13:30.669ZDEBGomicron-dev (store): query key key = _dendrite._tcp
115852025-04-30T06:13:30.669ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
115862025-04-30T06:13:30.669ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Fleet result = Ok(())
115872025-04-30T06:13:30.669ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50499 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1e7e9e35-69f5-4724-a2d7-04bcc6b96269
115882025-04-30T06:13:30.669ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database
115892025-04-30T06:13:30.681ZDEBGomicron-dev (BackgroundTasks): roles background_task = physical_disk_adoption roles = RoleSet { roles: {} }
115902025-04-30T06:13:30.681ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2475, tv_nsec: 443341374 } })
115912025-04-30T06:13:30.681ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = physical_disk_adoption resource = Database result = Ok(())
115922025-04-30T06:13:30.681ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
115932025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
115942025-04-30T06:13:30.682ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 26 17 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e &...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51197 req_id = 356e41b9-74be-41a1-967e-9279f681ed38
115952025-04-30T06:13:30.682ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 9751,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51197 req_id = 356e41b9-74be-41a1-967e-9279f681ed38
115962025-04-30T06:13:30.682ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
115972025-04-30T06:13:30.682ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51197 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 356e41b9-74be-41a1-967e-9279f681ed38
115982025-04-30T06:13:30.682ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
115992025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
116002025-04-30T06:13:30.682ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
116012025-04-30T06:13:30.682ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
116022025-04-30T06:13:30.682ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
116032025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
116042025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
116052025-04-30T06:13:30.682ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database
116062025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): roles background_task = region_snapshot_replacement_step roles = RoleSet { roles: {} }
116072025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = region_snapshot_replacement_step resource = Database result = Ok(())
116082025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_snapshot_replacement_start elapsed = 315.119505ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
116092025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116102025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bfc4639-5ca6-4323-a678-d4a0347a6fdf, lookup_type: ById(1bfc4639-5ca6-4323-a678-d4a0347a6fdf) } result = Ok(())
116112025-04-30T06:13:30.682ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
116122025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
116132025-04-30T06:13:30.682ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
116142025-04-30T06:13:30.682ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
116152025-04-30T06:13:30.683ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
116162025-04-30T06:13:30.683ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet
116172025-04-30T06:13:30.683ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = support_bundle_collector resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
116182025-04-30T06:13:30.683ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
116192025-04-30T06:13:30.683ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
116202025-04-30T06:13:30.683ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
116212025-04-30T06:13:30.683ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = f7d5534a-bde2-4482-8181-b20af3227c2a uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
116222025-04-30T06:13:30.683ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = f7d5534a-bde2-4482-8181-b20af3227c2a resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
116232025-04-30T06:13:30.683ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = f7d5534a-bde2-4482-8181-b20af3227c2a roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
116242025-04-30T06:13:30.683ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = f7d5534a-bde2-4482-8181-b20af3227c2a resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
116252025-04-30T06:13:30.706ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
116262025-04-30T06:13:30.706ZDEBGomicron-dev (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
116272025-04-30T06:13:30.706ZDEBGomicron-dev (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
116282025-04-30T06:13:30.706ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
116292025-04-30T06:13:30.709ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
116302025-04-30T06:13:30.714ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
116312025-04-30T06:13:30.714ZDEBGomicron-dev (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
116322025-04-30T06:13:30.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
116332025-04-30T06:13:30.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
116342025-04-30T06:13:30.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
116352025-04-30T06:13:30.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
116362025-04-30T06:13:30.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
116372025-04-30T06:13:30.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
116382025-04-30T06:13:30.726ZDEBGomicron-dev (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
116392025-04-30T06:13:30.726ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
116402025-04-30T06:13:30.731ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
116412025-04-30T06:13:30.731ZDEBGomicron-dev (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
116422025-04-30T06:13:30.735ZDEBGomicron-dev (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
116432025-04-30T06:13:30.735ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
116442025-04-30T06:13:30.736ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
116452025-04-30T06:13:30.738ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
116462025-04-30T06:13:30.798ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_replacement_driver elapsed = 458.914767ms iteration = 2 status = {"drive_invoked_ok":[],"errors":[],"finish_invoked_ok":[]}
116472025-04-30T06:13:30.798ZINFOomicron-dev (BackgroundTasks): listed instances with failed active VMMs background_task = instance_updater count = 0
116482025-04-30T06:13:30.798ZDEBGomicron-dev (BackgroundTasks): looking for instances with terminated active migrations... background_task = instance_updater
116492025-04-30T06:13:30.798ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database
116502025-04-30T06:13:30.798ZDEBGomicron-dev (BackgroundTasks): roles background_task = instance_updater roles = RoleSet { roles: {} }
116512025-04-30T06:13:30.799ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = instance_updater resource = Database result = Ok(())
116522025-04-30T06:13:30.808ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116532025-04-30T06:13:30.808ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116542025-04-30T06:13:30.813ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
116552025-04-30T06:13:30.814ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
116562025-04-30T06:13:30.814ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
116572025-04-30T06:13:30.814ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
116582025-04-30T06:13:30.815ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Fleet result = Ok(())
116592025-04-30T06:13:30.815ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
116602025-04-30T06:13:30.815ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
116612025-04-30T06:13:30.819ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
116622025-04-30T06:13:30.880ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_replacement elapsed = 542.949771ms iteration = 2 status = {"errors":[],"requests_completed_ok":[],"requests_created_ok":[],"start_invoked_ok":[]}
116632025-04-30T06:13:30.886ZINFOomicron-dev (dropshot_internal): request completed latency_us = 204775 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = f7d5534a-bde2-4482-8181-b20af3227c2a response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
116642025-04-30T06:13:30.923ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7d5534a-bde2-4482-8181-b20af3227c2a", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
116652025-04-30T06:13:30.923ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
116662025-04-30T06:13:30.926ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 678d2e3d-1f5f-4f02-9519-0c21537bda4f uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
116672025-04-30T06:13:30.926ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 678d2e3d-1f5f-4f02-9519-0c21537bda4f resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
116682025-04-30T06:13:30.926ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 678d2e3d-1f5f-4f02-9519-0c21537bda4f roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
116692025-04-30T06:13:30.927ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 678d2e3d-1f5f-4f02-9519-0c21537bda4f resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
116702025-04-30T06:13:30.945ZINFOomicron-dev (BackgroundTasks): all instance checks complete background_task = instance_watcher pruned_instances = 0 total_completed = 0 total_failed = 0 total_incomplete = 0 total_instances = 0 update_sagas_queued = 0
116712025-04-30T06:13:30.945ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = instance_watcher elapsed = 606.257438ms iteration = 2 status = {"failed_checks":{},"incomplete_checks":{},"instance_states":{},"pruned_instances":0,"total_instances":0,"update_sagas_queued":0}
116722025-04-30T06:13:31.009ZINFOomicron-dev (dropshot_internal): request completed latency_us = 83507 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 678d2e3d-1f5f-4f02-9519-0c21537bda4f response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
116732025-04-30T06:13:31.025ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "678d2e3d-1f5f-4f02-9519-0c21537bda4f", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
116742025-04-30T06:13:31.025ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
116752025-04-30T06:13:31.025ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
116762025-04-30T06:13:31.025ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
116772025-04-30T06:13:31.025ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
116782025-04-30T06:13:31.026ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
116792025-04-30T06:13:31.026ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
116802025-04-30T06:13:31.026ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
116812025-04-30T06:13:31.027ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
116822025-04-30T06:13:31.030ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116832025-04-30T06:13:31.036ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Fleet result = Ok(())
116842025-04-30T06:13:31.036ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database
116852025-04-30T06:13:31.036ZDEBGomicron-dev (BackgroundTasks): roles background_task = support_bundle_collector roles = RoleSet { roles: {} }
116862025-04-30T06:13:31.036ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = support_bundle_collector resource = Database result = Ok(())
116872025-04-30T06:13:31.090ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
116882025-04-30T06:13:31.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
116892025-04-30T06:13:31.091ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
116902025-04-30T06:13:31.091ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
116912025-04-30T06:13:31.095ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
116922025-04-30T06:13:31.098ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
116932025-04-30T06:13:31.098ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet
116942025-04-30T06:13:31.098ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = v2p_manager resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
116952025-04-30T06:13:31.098ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
116962025-04-30T06:13:31.098ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
116972025-04-30T06:13:31.099ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
116982025-04-30T06:13:31.099ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
116992025-04-30T06:13:31.099ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
117002025-04-30T06:13:31.102ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
117012025-04-30T06:13:31.102ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
117022025-04-30T06:13:31.108ZINFOomicron-dev (BackgroundTasks): listed instances with terminated active migrations background_task = instance_updater count = 0
117032025-04-30T06:13:31.109ZINFOomicron-dev (BackgroundTasks): instance updater activation completed background_task = instance_updater destroyed_active_vmms = 0 failed_active_vmms = 0 terminated_active_migrations = 0 update_sagas_completed = 0 update_sagas_started = 0
117042025-04-30T06:13:31.109ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = instance_updater elapsed = 769.849334ms iteration = 2 status = {"destroyed_active_vmms":0,"disabled":false,"failed_active_vmms":0,"query_errors":[],"saga_errors":[],"sagas_completed":0,"sagas_started":0,"terminated_active_migrations":0}
117052025-04-30T06:13:31.133ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117062025-04-30T06:13:31.138ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Fleet result = Ok(())
117072025-04-30T06:13:31.140ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database
117082025-04-30T06:13:31.140ZDEBGomicron-dev (BackgroundTasks): roles background_task = v2p_manager roles = RoleSet { roles: {} }
117092025-04-30T06:13:31.141ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = v2p_manager resource = Database result = Ok(())
117102025-04-30T06:13:31.141ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = region_snapshot_replacement_step elapsed = 805.931375ms iteration = 2 status = {"errors":[],"step_garbage_collect_invoked_ok":[],"step_invoked_ok":[],"step_records_created_ok":[],"step_set_volume_deleted_ok":[]}
117112025-04-30T06:13:31.197ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
117122025-04-30T06:13:31.228ZDEBGomicron-dev (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bfc4639-5ca6-4323-a678-d4a0347a6fdf original_time_created = 2025-04-30 06:13:06.783824 UTC target_id = 1bfc4639-5ca6-4323-a678-d4a0347a6fdf time_created = 2025-04-30 06:13:06.783824 UTC
117132025-04-30T06:13:31.228ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 888.406602ms iteration = 4 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1bfc4639-5ca6-4323-a678-d4a0347a6fdf","time_created":"2025-04-30T06:13:06.783824Z"}
117142025-04-30T06:13:31.229ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = physical_disk_adoption elapsed = 899.156871ms iteration = 3 status = {"physical_disks_added":0}
117152025-04-30T06:13:31.232ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
117162025-04-30T06:13:31.243ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
117172025-04-30T06:13:31.243ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117182025-04-30T06:13:31.244ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117192025-04-30T06:13:31.245ZINFOomicron-dev (BackgroundTasks): No bundles to collect background_task = support_bundle_collector
117202025-04-30T06:13:31.245ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = support_bundle_collector elapsed = 915.250887ms iteration = 2 status = {"cleanup_err":null,"cleanup_report":{"db_destroying_bundles_removed":0,"db_failing_bundles_updated":0,"sled_bundles_delete_failed":0,"sled_bundles_deleted_not_found":0,"sled_bundles_deleted_ok":0},"collection_err":null,"collection_report":null}
117212025-04-30T06:13:31.283ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
117222025-04-30T06:13:31.284ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117232025-04-30T06:13:31.284ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117242025-04-30T06:13:31.284ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
117252025-04-30T06:13:31.284ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117262025-04-30T06:13:31.408ZINFOomicron-dev (BackgroundTasks): VPC route manager sled b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager
117272025-04-30T06:13:31.408ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = None method = GET uri = http://[::1]:61881/vpc-routes
117282025-04-30T06:13:31.409ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:59418 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
117292025-04-30T06:13:31.409ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:59418 req_id = fe289329-73e7-449f-8098-2af320b55a4b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
117302025-04-30T06:13:31.410ZINFOomicron-dev (dropshot): request completed latency_us = 333 local_addr = [::1]:61881 method = GET remote_addr = [::1]:59418 req_id = fe289329-73e7-449f-8098-2af320b55a4b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /vpc-routes
117312025-04-30T06:13:31.410ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:61881/vpc-routes", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe289329-73e7-449f-8098-2af320b55a4b", "content-length": "2", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
117322025-04-30T06:13:31.410ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database
117332025-04-30T06:13:31.410ZDEBGomicron-dev (BackgroundTasks): roles background_task = vpc_route_manager roles = RoleSet { roles: {} }
117342025-04-30T06:13:31.411ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = vpc_route_manager resource = Database result = Ok(())
117352025-04-30T06:13:31.415ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager body = None method = GET uri = http://[::1]:61881/v2p
117362025-04-30T06:13:31.416ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:56801 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
117372025-04-30T06:13:31.416ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:56801 req_id = 2bae0859-86ec-49ca-bb36-1f08cb3bf1ef sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
117382025-04-30T06:13:31.416ZINFOomicron-dev (dropshot): request completed latency_us = 196 local_addr = [::1]:61881 method = GET remote_addr = [::1]:56801 req_id = 2bae0859-86ec-49ca-bb36-1f08cb3bf1ef response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /v2p
117392025-04-30T06:13:31.417ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = v2p_manager result = Ok(Response { url: "http://[::1]:61881/v2p", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bae0859-86ec-49ca-bb36-1f08cb3bf1ef", "content-length": "2", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
117402025-04-30T06:13:31.417ZINFOomicron-dev (BackgroundTasks): found opte v2p mappings background_task = v2p_manager interfaces = {} sled = sim-b6d65341
117412025-04-30T06:13:31.417ZINFOomicron-dev (BackgroundTasks): v2p mappings to delete background_task = v2p_manager mappings = [] sled = sim-b6d65341
117422025-04-30T06:13:31.417ZINFOomicron-dev (BackgroundTasks): v2p mappings to add background_task = v2p_manager mappings = [] sled = sim-b6d65341
117432025-04-30T06:13:31.420ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = v2p_manager elapsed = 1.088172672s iteration = 2 status = {}
117442025-04-30T06:13:31.423ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) }
117452025-04-30T06:13:31.423ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117462025-04-30T06:13:31.424ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = service_zone_nat_tracker resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117472025-04-30T06:13:31.424ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database
117482025-04-30T06:13:31.424ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {} }
117492025-04-30T06:13:31.425ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Database result = Ok(())
117502025-04-30T06:13:31.429ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
117512025-04-30T06:13:31.429ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117522025-04-30T06:13:31.429ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
117532025-04-30T06:13:31.429ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117542025-04-30T06:13:31.429ZDEBGomicron-dev (BackgroundTasks): roles background_task = service_zone_nat_tracker roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117552025-04-30T06:13:31.441ZINFOomicron-dev (BackgroundTasks): computed internet gateway mappings for sled assocs = {} background_task = vpc_route_manager sled = sim-b6d65341
117562025-04-30T06:13:31.441ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager body = Some(Body) method = PUT uri = http://[::1]:61881/eip-gateways
117572025-04-30T06:13:31.442ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = PUT remote_addr = [::1]:59418 req_id = a6accece-4a95-4389-a0c2-34763400d8ae sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
117582025-04-30T06:13:31.443ZINFOomicron-dev (dropshot): request completed latency_us = 373 local_addr = [::1]:61881 method = PUT remote_addr = [::1]:59418 req_id = a6accece-4a95-4389-a0c2-34763400d8ae response_code = 204 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /eip-gateways
117592025-04-30T06:13:31.443ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = b6d65341-167c-41df-9b5c-41cded99c229 background_task = vpc_route_manager result = Ok(Response { url: "http://[::1]:61881/eip-gateways", status: 204, headers: {"x-request-id": "a6accece-4a95-4389-a0c2-34763400d8ae", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
117602025-04-30T06:13:31.443ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = service_zone_nat_tracker resource = Sled { parent: Fleet, key: b6d65341-167c-41df-9b5c-41cded99c229, lookup_type: ById(b6d65341-167c-41df-9b5c-41cded99c229) } result = Ok(())
117612025-04-30T06:13:31.444ZERROomicron-dev (BackgroundTasks): ipv6 addresses for service zone nat not implemented background_task = service_zone_nat_tracker
117622025-04-30T06:13:31.445ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = vpc_route_manager elapsed = 1.108996833s iteration = 2 status = {}
117632025-04-30T06:13:31.447ZERROomicron-dev (BackgroundTasks): generated config for fewer than the minimum allowed number of dns zones background_task = service_zone_nat_tracker
117642025-04-30T06:13:31.448ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = service_zone_nat_tracker elapsed = 1.10966776s iteration = 2 status = {"error":"generated config for fewer than the minimum allowed number of dns zones"}
117652025-04-30T06:13:31.452ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
117662025-04-30T06:13:31.452ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
117672025-04-30T06:13:31.453ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
117682025-04-30T06:13:31.471ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
117692025-04-30T06:13:31.472ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
117702025-04-30T06:13:31.473ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:32797 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
117712025-04-30T06:13:31.474ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:32797 req_id = aa150bfe-fd3f-4aae-82ed-1864762b884e sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
117722025-04-30T06:13:31.475ZINFOomicron-dev (dropshot): request completed latency_us = 467 local_addr = [::1]:61881 method = GET remote_addr = [::1]:32797 req_id = aa150bfe-fd3f-4aae-82ed-1864762b884e response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
117732025-04-30T06:13:31.477ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa150bfe-fd3f-4aae-82ed-1864762b884e", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
117742025-04-30T06:13:31.481ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
117752025-04-30T06:13:31.481ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
117762025-04-30T06:13:31.481ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
117772025-04-30T06:13:31.485ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
117782025-04-30T06:13:31.485ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117792025-04-30T06:13:31.485ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117802025-04-30T06:13:31.485ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117812025-04-30T06:13:31.487ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117822025-04-30T06:13:31.491ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117832025-04-30T06:13:31.496ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
117842025-04-30T06:13:31.500ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
117852025-04-30T06:13:31.500ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117862025-04-30T06:13:31.501ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117872025-04-30T06:13:31.502ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117882025-04-30T06:13:31.540ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
117892025-04-30T06:13:31.541ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
117902025-04-30T06:13:31.541ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 1.271459991s iteration = 41 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
117912025-04-30T06:13:31.541ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 42 reason = Signaled
117922025-04-30T06:13:31.541ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
117932025-04-30T06:13:31.541ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
117942025-04-30T06:13:31.541ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
117952025-04-30T06:13:31.541ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
117962025-04-30T06:13:31.542ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
117972025-04-30T06:13:31.545ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
117982025-04-30T06:13:31.549ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
117992025-04-30T06:13:31.549ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
118002025-04-30T06:13:31.549ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118012025-04-30T06:13:31.549ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
118022025-04-30T06:13:31.549ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118032025-04-30T06:13:31.559ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
118042025-04-30T06:13:31.560ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
118052025-04-30T06:13:31.560ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
118062025-04-30T06:13:31.562ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
118072025-04-30T06:13:31.565ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118082025-04-30T06:13:31.567ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
118092025-04-30T06:13:31.567ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
118102025-04-30T06:13:31.568ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 13 ff 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:65279 req_id = fa2c5d47-7dd8-4a7b-b89f-1391777d60e4
118112025-04-30T06:13:31.568ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5119,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65279 req_id = fa2c5d47-7dd8-4a7b-b89f-1391777d60e4
118122025-04-30T06:13:31.568ZDEBGomicron-dev (store): query key key = _mgs._tcp
118132025-04-30T06:13:31.568ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
118142025-04-30T06:13:31.568ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65279 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = fa2c5d47-7dd8-4a7b-b89f-1391777d60e4
118152025-04-30T06:13:31.569ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2476, tv_nsec: 368719359 } })
118162025-04-30T06:13:31.569ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
118172025-04-30T06:13:31.570ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 1f e0 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53564 req_id = dfbb2fbc-0354-460a-8ef3-ccd9d26079bd
118182025-04-30T06:13:31.570ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8160,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53564 req_id = dfbb2fbc-0354-460a-8ef3-ccd9d26079bd
118192025-04-30T06:13:31.570ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
118202025-04-30T06:13:31.570ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53564 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = dfbb2fbc-0354-460a-8ef3-ccd9d26079bd
118212025-04-30T06:13:31.571ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
118222025-04-30T06:13:31.572ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118232025-04-30T06:13:31.572ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
118242025-04-30T06:13:31.572ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118252025-04-30T06:13:31.575ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118262025-04-30T06:13:31.575ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
118272025-04-30T06:13:31.575ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118282025-04-30T06:13:31.609ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
118292025-04-30T06:13:31.633ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
118302025-04-30T06:13:31.634ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 34 30 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c 40..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58082 req_id = aa121697-024d-438e-a2c8-8962eadc8a7f
118312025-04-30T06:13:31.634ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13360,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58082 req_id = aa121697-024d-438e-a2c8-8962eadc8a7f
118322025-04-30T06:13:31.634ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
118332025-04-30T06:13:31.635ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 13360, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58082 req_id = aa121697-024d-438e-a2c8-8962eadc8a7f
118342025-04-30T06:13:31.635ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
118352025-04-30T06:13:31.635ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
118362025-04-30T06:13:31.635ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
118372025-04-30T06:13:31.638ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1a9a3f0-855d-4ab0-9a1d-19f3a21bed6f", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
118382025-04-30T06:13:31.638ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
118392025-04-30T06:13:31.641ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03b61df4-236d-4083-8fb3-611d500ab16a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118402025-04-30T06:13:31.641ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
118412025-04-30T06:13:31.646ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "16f554d0-02c6-4443-885d-90f72cc5a04d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118422025-04-30T06:13:31.646ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
118432025-04-30T06:13:31.650ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "74cb9bca-d07c-4b8c-863f-a348208bfb99", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118442025-04-30T06:13:31.650ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
118452025-04-30T06:13:31.655ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ebac273c-d4f0-413f-bab2-e8c69039a4d4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118462025-04-30T06:13:31.657ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
118472025-04-30T06:13:31.661ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
118482025-04-30T06:13:31.661ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
118492025-04-30T06:13:31.661ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118502025-04-30T06:13:31.661ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
118512025-04-30T06:13:31.662ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118522025-04-30T06:13:31.665ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "678046c9-5457-438d-85be-bc8a78de5404", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
118532025-04-30T06:13:31.666ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
118542025-04-30T06:13:31.668ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac320f49-af84-4c27-aae7-1fd04178cdcd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
118552025-04-30T06:13:31.668ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
118562025-04-30T06:13:31.670ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "645515bb-41b0-45d5-8145-a6e6faa59714", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
118572025-04-30T06:13:31.670ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
118582025-04-30T06:13:31.671ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f722a06d-7168-4337-82c1-2d5a4777ca78", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
118592025-04-30T06:13:31.671ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
118602025-04-30T06:13:31.674ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "18203de1-4bb8-4863-b7a1-efc5cefe929f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
118612025-04-30T06:13:31.674ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
118622025-04-30T06:13:31.675ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "635e1920-bff9-40d7-9249-16d633760018", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
118632025-04-30T06:13:31.675ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
118642025-04-30T06:13:31.676ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d62a0af2-7522-42dd-8168-010550912d0b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
118652025-04-30T06:13:31.676ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
118662025-04-30T06:13:31.677ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
118672025-04-30T06:13:31.677ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6a1d546-e6cb-4a11-b9e9-9eb7fa12ecfe", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
118682025-04-30T06:13:31.677ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
118692025-04-30T06:13:31.679ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cfc81d44-864d-4bae-8f1b-b357ea0f4f05", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118702025-04-30T06:13:31.679ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
118712025-04-30T06:13:31.681ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "68208615-878f-414d-952a-768022f226fb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118722025-04-30T06:13:31.684ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
118732025-04-30T06:13:31.684ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3693f54e-4b31-4b14-885d-034fd98de593", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118742025-04-30T06:13:31.684ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
118752025-04-30T06:13:31.685ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a91fc2e-4c0e-4312-bd84-dd7aacb26c28", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118762025-04-30T06:13:31.685ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
118772025-04-30T06:13:31.687ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
118782025-04-30T06:13:31.687ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118792025-04-30T06:13:31.687ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
118802025-04-30T06:13:31.687ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118812025-04-30T06:13:31.687ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3908957-c414-42e9-aa9a-70b3eb995056", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118822025-04-30T06:13:31.687ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
118832025-04-30T06:13:31.694ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
118842025-04-30T06:13:31.694ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
118852025-04-30T06:13:31.697ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
118862025-04-30T06:13:31.701ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fa8bc5b-b393-4d99-ace1-e05fc09db664", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118872025-04-30T06:13:31.702ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
118882025-04-30T06:13:31.705ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0af42f71-5487-4700-a1e3-f8ab461e1df8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118892025-04-30T06:13:31.705ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
118902025-04-30T06:13:31.706ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f64e7418-2897-4441-8690-791d1c8936a0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118912025-04-30T06:13:31.707ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
118922025-04-30T06:13:31.714ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3339417d-72a9-46ba-957d-268d009c01b6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118932025-04-30T06:13:31.714ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
118942025-04-30T06:13:31.717ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a1479a3-2ecd-45ec-9a2b-b8fe81151903", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118952025-04-30T06:13:31.717ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
118962025-04-30T06:13:31.731ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118972025-04-30T06:13:31.731ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
118982025-04-30T06:13:31.733ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "99fd0635-5842-4ca4-bc1c-dfbde26e0c7e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
118992025-04-30T06:13:31.734ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
119002025-04-30T06:13:31.736ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
119012025-04-30T06:13:31.743ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cd321a0-fafa-4d3e-b8d9-dd7c66d86a65", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119022025-04-30T06:13:31.745ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
119032025-04-30T06:13:31.754ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
119042025-04-30T06:13:31.754ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
119052025-04-30T06:13:31.754ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
119062025-04-30T06:13:31.757ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
119072025-04-30T06:13:31.758ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "805c66b1-d2a6-4f71-8a21-e744c00922ca", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119082025-04-30T06:13:31.758ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
119092025-04-30T06:13:31.761ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e690385e-6b66-4c32-863b-e07b37b8f836", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
119102025-04-30T06:13:31.761ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
119112025-04-30T06:13:31.763ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c351e1a-2804-4541-8100-721f2180b77d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
119122025-04-30T06:13:31.764ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
119132025-04-30T06:13:31.767ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d3dfb38-5ec3-4b40-ba10-c9f6bbd67617", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
119142025-04-30T06:13:31.767ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
119152025-04-30T06:13:31.769ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6e9c514-aed4-4b36-b22d-e78b3fcef301", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
119162025-04-30T06:13:31.770ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
119172025-04-30T06:13:31.771ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5119387e-7451-4321-9593-206681995a47", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
119182025-04-30T06:13:31.772ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
119192025-04-30T06:13:31.773ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a53dfc43-86d1-4606-9fe6-8eb8f276144f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:30 GMT"} })
119202025-04-30T06:13:31.773ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
119212025-04-30T06:13:31.775ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "446fd281-17b7-456a-8396-be28e645e51e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119222025-04-30T06:13:31.776ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
119232025-04-30T06:13:31.777ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d25f5584-1e27-45c0-acb7-73c1033df707", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119242025-04-30T06:13:31.777ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
119252025-04-30T06:13:31.778ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e934a57a-51b5-41e5-a377-493ba82f3d95", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119262025-04-30T06:13:31.779ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
119272025-04-30T06:13:31.782ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6c2281d8-10ba-4fed-a240-e6ede9c21999", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119282025-04-30T06:13:31.782ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
119292025-04-30T06:13:31.786ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "69a4dea0-c7ce-4638-9431-727a05bbd04a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119302025-04-30T06:13:31.786ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
119312025-04-30T06:13:31.790ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf4f1e47-d1ba-4617-9c27-d7335346f928", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119322025-04-30T06:13:31.791ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
119332025-04-30T06:13:31.796ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b58f82b-2623-4151-a381-9d696e7d1697", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119342025-04-30T06:13:31.796ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
119352025-04-30T06:13:31.799ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8c9c40d-92f4-4d22-9582-c7ea62968a7f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119362025-04-30T06:13:31.799ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
119372025-04-30T06:13:31.804ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9267b55-aa95-4d20-895e-3ba72654b082", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119382025-04-30T06:13:31.805ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
119392025-04-30T06:13:31.806ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5936d749-39da-4c51-ad8b-6932e2f258c5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119402025-04-30T06:13:31.807ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
119412025-04-30T06:13:31.808ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
119422025-04-30T06:13:31.808ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119432025-04-30T06:13:31.808ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119442025-04-30T06:13:31.808ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
119452025-04-30T06:13:31.809ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119462025-04-30T06:13:31.811ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ade77f6-41e0-4afb-a6cf-b0b2365d417f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119472025-04-30T06:13:31.812ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
119482025-04-30T06:13:31.813ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119492025-04-30T06:13:31.814ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce1da66c-9b2f-4004-831e-ccbf869c3da8", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119502025-04-30T06:13:31.814ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
119512025-04-30T06:13:31.816ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f6df166-f0ea-41c1-99f3-6df2c6c155bc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119522025-04-30T06:13:31.816ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
119532025-04-30T06:13:31.816ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119542025-04-30T06:13:31.816ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119552025-04-30T06:13:31.816ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119562025-04-30T06:13:31.817ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119572025-04-30T06:13:31.818ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
119582025-04-30T06:13:31.818ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119592025-04-30T06:13:31.818ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
119602025-04-30T06:13:31.818ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119612025-04-30T06:13:31.822ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119622025-04-30T06:13:31.830ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119632025-04-30T06:13:31.830ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
119642025-04-30T06:13:31.830ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119652025-04-30T06:13:31.833ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
119662025-04-30T06:13:31.833ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
119672025-04-30T06:13:31.833ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119682025-04-30T06:13:31.833ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119692025-04-30T06:13:31.833ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119702025-04-30T06:13:31.833ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119712025-04-30T06:13:31.841ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119722025-04-30T06:13:31.849ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
119732025-04-30T06:13:31.849ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119742025-04-30T06:13:31.849ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119752025-04-30T06:13:31.850ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119762025-04-30T06:13:31.862ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
119772025-04-30T06:13:31.915ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
119782025-04-30T06:13:31.915ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
119792025-04-30T06:13:31.915ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
119802025-04-30T06:13:31.915ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119812025-04-30T06:13:31.915ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119822025-04-30T06:13:31.915ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
119832025-04-30T06:13:31.916ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:58882 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
119842025-04-30T06:13:31.916ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:58882 req_id = 3ec6428a-cb42-4335-ba6b-1067bccc5fd6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
119852025-04-30T06:13:31.916ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
119862025-04-30T06:13:31.917ZINFOomicron-dev (dropshot): request completed latency_us = 580 local_addr = [::1]:61881 method = GET remote_addr = [::1]:58882 req_id = 3ec6428a-cb42-4335-ba6b-1067bccc5fd6 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
119872025-04-30T06:13:31.917ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "3ec6428a-cb42-4335-ba6b-1067bccc5fd6", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
119882025-04-30T06:13:31.920ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
119892025-04-30T06:13:31.920ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
119902025-04-30T06:13:31.920ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
119912025-04-30T06:13:31.921ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
119922025-04-30T06:13:31.923ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
119932025-04-30T06:13:31.923ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
119942025-04-30T06:13:31.923ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
119952025-04-30T06:13:31.923ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
119962025-04-30T06:13:31.924ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
119972025-04-30T06:13:31.929ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
119982025-04-30T06:13:31.929ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
119992025-04-30T06:13:31.929ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
120002025-04-30T06:13:31.930ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120012025-04-30T06:13:31.931ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120022025-04-30T06:13:31.934ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120032025-04-30T06:13:31.934ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
120042025-04-30T06:13:31.934ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120052025-04-30T06:13:31.947ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
120062025-04-30T06:13:31.947ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
120072025-04-30T06:13:31.947ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120082025-04-30T06:13:31.949ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
120092025-04-30T06:13:32.023ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
120102025-04-30T06:13:32.027ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
120112025-04-30T06:13:32.027ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
120122025-04-30T06:13:32.027ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 486.209971ms iteration = 42 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
120132025-04-30T06:13:32.027ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 43 reason = Signaled
120142025-04-30T06:13:32.027ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
120152025-04-30T06:13:32.027ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
120162025-04-30T06:13:32.027ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
120172025-04-30T06:13:32.027ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120182025-04-30T06:13:32.028ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
120192025-04-30T06:13:32.031ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120202025-04-30T06:13:32.036ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
120212025-04-30T06:13:32.036ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
120222025-04-30T06:13:32.037ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
120232025-04-30T06:13:32.037ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
120242025-04-30T06:13:32.042ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
120252025-04-30T06:13:32.042ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
120262025-04-30T06:13:32.043ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 96 0f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59533 req_id = bad18e77-0033-4d38-a043-23c0a87641b7
120272025-04-30T06:13:32.044ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 38415,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59533 req_id = bad18e77-0033-4d38-a043-23c0a87641b7
120282025-04-30T06:13:32.044ZDEBGomicron-dev (store): query key key = _mgs._tcp
120292025-04-30T06:13:32.044ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
120302025-04-30T06:13:32.045ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59533 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = bad18e77-0033-4d38-a043-23c0a87641b7
120312025-04-30T06:13:32.045ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2476, tv_nsec: 844559529 } })
120322025-04-30T06:13:32.045ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
120332025-04-30T06:13:32.046ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: fb c4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59067 req_id = 630c2374-a979-4ec9-9242-8a8d047adc5a
120342025-04-30T06:13:32.046ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64452,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59067 req_id = 630c2374-a979-4ec9-9242-8a8d047adc5a
120352025-04-30T06:13:32.046ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
120362025-04-30T06:13:32.046ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59067 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 630c2374-a979-4ec9-9242-8a8d047adc5a
120372025-04-30T06:13:32.112ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
120382025-04-30T06:13:32.114ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
120392025-04-30T06:13:32.114ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
120402025-04-30T06:13:32.114ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120412025-04-30T06:13:32.114ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
120422025-04-30T06:13:32.115ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
120432025-04-30T06:13:32.115ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
120442025-04-30T06:13:32.115ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
120452025-04-30T06:13:32.116ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120462025-04-30T06:13:32.117ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: bd 9f 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62044 req_id = 29d92f30-913e-487b-86f4-f8bb85153629
120472025-04-30T06:13:32.118ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 48543,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62044 req_id = 29d92f30-913e-487b-86f4-f8bb85153629
120482025-04-30T06:13:32.119ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
120492025-04-30T06:13:32.123ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
120502025-04-30T06:13:32.123ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 48543, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62044 req_id = 29d92f30-913e-487b-86f4-f8bb85153629
120512025-04-30T06:13:32.123ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
120522025-04-30T06:13:32.123ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
120532025-04-30T06:13:32.124ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
120542025-04-30T06:13:32.124ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
120552025-04-30T06:13:32.126ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
120562025-04-30T06:13:32.126ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
120572025-04-30T06:13:32.128ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f96f208-207d-4478-9389-2130c7df4c28", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120582025-04-30T06:13:32.129ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
120592025-04-30T06:13:32.132ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a170018-08b8-4f0e-b7a4-243c0c18cbd5", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120602025-04-30T06:13:32.132ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
120612025-04-30T06:13:32.136ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d85ae8f7-8a6e-421f-ad06-e61c68535171", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120622025-04-30T06:13:32.137ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
120632025-04-30T06:13:32.141ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ecad37e-1e42-4566-8060-a825a59ec138", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120642025-04-30T06:13:32.141ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
120652025-04-30T06:13:32.141ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
120662025-04-30T06:13:32.141ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
120672025-04-30T06:13:32.144ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
120682025-04-30T06:13:32.144ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120692025-04-30T06:13:32.144ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
120702025-04-30T06:13:32.145ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120712025-04-30T06:13:32.147ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5841f56-bcd3-488d-973d-49807ce5a307", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120722025-04-30T06:13:32.148ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
120732025-04-30T06:13:32.149ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
120742025-04-30T06:13:32.149ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
120752025-04-30T06:13:32.151ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
120762025-04-30T06:13:32.154ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6feccd1f-1867-4266-9d5b-dbac678f5783", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120772025-04-30T06:13:32.155ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
120782025-04-30T06:13:32.159ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcfb6084-7a5a-4be8-861f-ffccf1777bf9", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120792025-04-30T06:13:32.159ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
120802025-04-30T06:13:32.164ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c878148-db60-4200-9eb9-be2a6289ae3b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120812025-04-30T06:13:32.164ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
120822025-04-30T06:13:32.165ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "47e233c0-2434-46d4-9bc8-0f396e1adf76", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120832025-04-30T06:13:32.165ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
120842025-04-30T06:13:32.166ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d3f78a6-185a-4103-918c-bae394efdc3e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120852025-04-30T06:13:32.166ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
120862025-04-30T06:13:32.167ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "59bd5712-f916-4ec3-8051-bc225c54f667", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120872025-04-30T06:13:32.167ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
120882025-04-30T06:13:32.168ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "21cee1ff-4e45-48a7-8cd3-4ac986f548b5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120892025-04-30T06:13:32.168ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
120902025-04-30T06:13:32.171ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "697810e1-909e-4527-8038-cb603e5a5795", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120912025-04-30T06:13:32.171ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
120922025-04-30T06:13:32.172ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef1e3b15-73e8-4e6f-82d8-b9ec773337ce", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120932025-04-30T06:13:32.172ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
120942025-04-30T06:13:32.175ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1f6a29d-c379-4760-b629-1801e604aaf7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120952025-04-30T06:13:32.175ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
120962025-04-30T06:13:32.176ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "999d4c1a-2ab6-4549-a3be-479c79b829ef", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120972025-04-30T06:13:32.176ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
120982025-04-30T06:13:32.177ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eab87b18-b65c-428a-a3b7-52a21c61e799", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
120992025-04-30T06:13:32.177ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
121002025-04-30T06:13:32.181ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "44a65134-4a33-417d-ac0b-e71719172e91", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121012025-04-30T06:13:32.181ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
121022025-04-30T06:13:32.182ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "da64045f-5bf3-4e52-aa2c-11280e071697", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121032025-04-30T06:13:32.182ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
121042025-04-30T06:13:32.183ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "95775440-682e-4c31-9a02-34c4b7e345e2", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121052025-04-30T06:13:32.183ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
121062025-04-30T06:13:32.185ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14f82708-27ee-48f9-bbdc-27fa1d544dd3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121072025-04-30T06:13:32.185ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
121082025-04-30T06:13:32.191ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6427b69-c734-45de-90e1-9e06929acb23", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121092025-04-30T06:13:32.191ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
121102025-04-30T06:13:32.196ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
121112025-04-30T06:13:32.212ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6c1ce38-8b12-4123-ba12-fe58ca22399c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121122025-04-30T06:13:32.213ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
121132025-04-30T06:13:32.217ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d55634aa-d4ef-4b88-9304-0cce74989d19", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121142025-04-30T06:13:32.217ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
121152025-04-30T06:13:32.218ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
121162025-04-30T06:13:32.220ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfe59007-bdaf-4f20-96a4-3eea3b11d1c3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121172025-04-30T06:13:32.221ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
121182025-04-30T06:13:32.223ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dc34ff7-2661-4a87-b30f-4f88139badef", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121192025-04-30T06:13:32.223ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
121202025-04-30T06:13:32.225ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0e5f965-d293-484f-a0cb-5a003a1e669f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121212025-04-30T06:13:32.225ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
121222025-04-30T06:13:32.227ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "067d5f05-89b5-4bba-b48f-f02a40491f85", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121232025-04-30T06:13:32.227ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
121242025-04-30T06:13:32.230ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0627c98b-4fcf-41c6-b0ff-60395a808bdd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121252025-04-30T06:13:32.230ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
121262025-04-30T06:13:32.244ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7675bed7-0730-40ef-aeae-69b6edd4221c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121272025-04-30T06:13:32.244ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
121282025-04-30T06:13:32.247ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "efd16e4e-8058-4c1a-b071-2a959c15ad2b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121292025-04-30T06:13:32.249ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
121302025-04-30T06:13:32.252ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
121312025-04-30T06:13:32.252ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121322025-04-30T06:13:32.253ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121332025-04-30T06:13:32.253ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
121342025-04-30T06:13:32.256ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121352025-04-30T06:13:32.259ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121362025-04-30T06:13:32.267ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
121372025-04-30T06:13:32.267ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121382025-04-30T06:13:32.267ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
121392025-04-30T06:13:32.268ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121402025-04-30T06:13:32.271ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121412025-04-30T06:13:32.271ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
121422025-04-30T06:13:32.272ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121432025-04-30T06:13:32.273ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc556638-e723-4f66-9dbc-89c4f660a758", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121442025-04-30T06:13:32.273ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
121452025-04-30T06:13:32.276ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7141bc1-bff4-41de-b871-37bf0a56435e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121462025-04-30T06:13:32.276ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
121472025-04-30T06:13:32.277ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "16db8288-93c2-4f6d-b919-09c781e364f3", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121482025-04-30T06:13:32.277ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
121492025-04-30T06:13:32.279ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eae2d765-47e5-4968-9d78-d122cd8b1771", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121502025-04-30T06:13:32.280ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
121512025-04-30T06:13:32.285ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bb3929e-eaec-42ca-b8cd-78a899fcbaa9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121522025-04-30T06:13:32.285ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
121532025-04-30T06:13:32.289ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d4afa35-17ac-4f5a-a5db-a1b1c999aa93", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121542025-04-30T06:13:32.289ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
121552025-04-30T06:13:32.292ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9740381-c13d-425c-b3b9-c889e0121739", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121562025-04-30T06:13:32.293ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
121572025-04-30T06:13:32.304ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e841059-54d1-4ab8-bec0-6ffbe8e3f7c5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121582025-04-30T06:13:32.305ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
121592025-04-30T06:13:32.317ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cfb5e5f-1c87-41a2-abe2-f84d43f8d0c2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121602025-04-30T06:13:32.317ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
121612025-04-30T06:13:32.322ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
121622025-04-30T06:13:32.333ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "675531b7-a90f-440b-aef2-652b2b12c9d6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121632025-04-30T06:13:32.333ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
121642025-04-30T06:13:32.334ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "be00c31c-e62c-48ed-be31-eb299ccbeffe", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121652025-04-30T06:13:32.334ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
121662025-04-30T06:13:32.335ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c43ffa8-1e13-4df9-b6e3-5dad5fb01da2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121672025-04-30T06:13:32.336ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
121682025-04-30T06:13:32.337ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b350e768-b4cd-4fd4-bc56-5ed6cfa2133a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121692025-04-30T06:13:32.338ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
121702025-04-30T06:13:32.339ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa6a6a2c-10ab-4ecc-997b-fa9575fb0ccc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
121712025-04-30T06:13:32.339ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
121722025-04-30T06:13:32.339ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121732025-04-30T06:13:32.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
121742025-04-30T06:13:32.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121752025-04-30T06:13:32.341ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
121762025-04-30T06:13:32.345ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121772025-04-30T06:13:32.349ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
121782025-04-30T06:13:32.349ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
121792025-04-30T06:13:32.349ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121802025-04-30T06:13:32.349ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
121812025-04-30T06:13:32.349ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121822025-04-30T06:13:32.350ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
121832025-04-30T06:13:32.353ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121842025-04-30T06:13:32.360ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
121852025-04-30T06:13:32.360ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
121862025-04-30T06:13:32.360ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
121872025-04-30T06:13:32.362ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
121882025-04-30T06:13:32.374ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
121892025-04-30T06:13:32.374ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
121902025-04-30T06:13:32.374ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
121912025-04-30T06:13:32.374ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
121922025-04-30T06:13:32.375ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
121932025-04-30T06:13:32.462ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
121942025-04-30T06:13:32.462ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
121952025-04-30T06:13:32.462ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:55189 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
121962025-04-30T06:13:32.464ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:55189 req_id = 4d16ccfa-2590-4bf3-8246-0eda8b55ad3a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
121972025-04-30T06:13:32.465ZINFOomicron-dev (dropshot): request completed latency_us = 1191 local_addr = [::1]:61881 method = GET remote_addr = [::1]:55189 req_id = 4d16ccfa-2590-4bf3-8246-0eda8b55ad3a response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
121982025-04-30T06:13:32.467ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
121992025-04-30T06:13:32.467ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d16ccfa-2590-4bf3-8246-0eda8b55ad3a", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
122002025-04-30T06:13:32.468ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
122012025-04-30T06:13:32.468ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
122022025-04-30T06:13:32.468ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
122032025-04-30T06:13:32.471ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
122042025-04-30T06:13:32.471ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122052025-04-30T06:13:32.471ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122062025-04-30T06:13:32.471ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122072025-04-30T06:13:32.472ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122082025-04-30T06:13:32.475ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122092025-04-30T06:13:32.475ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
122102025-04-30T06:13:32.476ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122112025-04-30T06:13:32.476ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
122122025-04-30T06:13:32.477ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122132025-04-30T06:13:32.483ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
122142025-04-30T06:13:32.483ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122152025-04-30T06:13:32.483ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122162025-04-30T06:13:32.484ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122172025-04-30T06:13:32.486ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122182025-04-30T06:13:32.486ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
122192025-04-30T06:13:32.488ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122202025-04-30T06:13:32.527ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
122212025-04-30T06:13:32.527ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
122222025-04-30T06:13:32.532ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
122232025-04-30T06:13:32.532ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 503.096066ms iteration = 43 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
122242025-04-30T06:13:32.532ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 44 reason = Signaled
122252025-04-30T06:13:32.533ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
122262025-04-30T06:13:32.533ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122272025-04-30T06:13:32.533ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122282025-04-30T06:13:32.533ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122292025-04-30T06:13:32.533ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122302025-04-30T06:13:32.546ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122312025-04-30T06:13:32.562ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
122322025-04-30T06:13:32.562ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
122332025-04-30T06:13:32.562ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
122342025-04-30T06:13:32.563ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
122352025-04-30T06:13:32.567ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
122362025-04-30T06:13:32.567ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
122372025-04-30T06:13:32.567ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 18 df 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:57211 req_id = 4454e0c5-9307-4332-8b97-0d2397108a33
122382025-04-30T06:13:32.567ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6367,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57211 req_id = 4454e0c5-9307-4332-8b97-0d2397108a33
122392025-04-30T06:13:32.568ZDEBGomicron-dev (store): query key key = _mgs._tcp
122402025-04-30T06:13:32.568ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122412025-04-30T06:13:32.568ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57211 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 4454e0c5-9307-4332-8b97-0d2397108a33
122422025-04-30T06:13:32.569ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2477, tv_nsec: 368203165 } })
122432025-04-30T06:13:32.569ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
122442025-04-30T06:13:32.570ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6c e7 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e l...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60441 req_id = 75558711-c37a-4e04-868f-e6186d60744d
122452025-04-30T06:13:32.570ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27879,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60441 req_id = 75558711-c37a-4e04-868f-e6186d60744d
122462025-04-30T06:13:32.570ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122472025-04-30T06:13:32.570ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60441 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 75558711-c37a-4e04-868f-e6186d60744d
122482025-04-30T06:13:32.697ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
122492025-04-30T06:13:32.699ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
122502025-04-30T06:13:32.699ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
122512025-04-30T06:13:32.699ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
122522025-04-30T06:13:32.699ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
122532025-04-30T06:13:32.699ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122542025-04-30T06:13:32.699ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
122552025-04-30T06:13:32.699ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 4e859529-f254-48f2-b85b-cdef37c52bfc uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
122562025-04-30T06:13:32.700ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 21 8b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c !............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54188 req_id = ecb07bb2-b42f-4276-8602-eb42161acf75
122572025-04-30T06:13:32.700ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 4e859529-f254-48f2-b85b-cdef37c52bfc resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
122582025-04-30T06:13:32.700ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122592025-04-30T06:13:32.700ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 4e859529-f254-48f2-b85b-cdef37c52bfc roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
122602025-04-30T06:13:32.700ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8587,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54188 req_id = ecb07bb2-b42f-4276-8602-eb42161acf75
122612025-04-30T06:13:32.700ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
122622025-04-30T06:13:32.701ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 8587, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54188 req_id = ecb07bb2-b42f-4276-8602-eb42161acf75
122632025-04-30T06:13:32.701ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 4e859529-f254-48f2-b85b-cdef37c52bfc resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
122642025-04-30T06:13:32.701ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
122652025-04-30T06:13:32.701ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
122662025-04-30T06:13:32.701ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
122672025-04-30T06:13:32.704ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "e962c412-c6b3-4bc5-ba8f-ecfdc97824ca", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
122682025-04-30T06:13:32.705ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
122692025-04-30T06:13:32.705ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
122702025-04-30T06:13:32.708ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4037a60-628b-4fa0-bb2e-c61a77139fc3", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
122712025-04-30T06:13:32.708ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
122722025-04-30T06:13:32.711ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
122732025-04-30T06:13:32.712ZINFOomicron-dev (dropshot_internal): request completed latency_us = 12216 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 4e859529-f254-48f2-b85b-cdef37c52bfc response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
122742025-04-30T06:13:32.712ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e859529-f254-48f2-b85b-cdef37c52bfc", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
122752025-04-30T06:13:32.712ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
122762025-04-30T06:13:32.713ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 2b 4e 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 +N..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54100 req_id = 7fecd282-0c20-4d05-b7bd-dd405be01614
122772025-04-30T06:13:32.713ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 11086,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54100 req_id = 7fecd282-0c20-4d05-b7bd-dd405be01614
122782025-04-30T06:13:32.713ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = cd956c47-f550-4e24-a4ec-59f3e7fa2f30 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
122792025-04-30T06:13:32.713ZDEBGomicron-dev (store): query key key = _dendrite._tcp
122802025-04-30T06:13:32.713ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122812025-04-30T06:13:32.713ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54100 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 7fecd282-0c20-4d05-b7bd-dd405be01614
122822025-04-30T06:13:32.714ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
122832025-04-30T06:13:32.714ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122842025-04-30T06:13:32.714ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
122852025-04-30T06:13:32.714ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
122862025-04-30T06:13:32.715ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = cd956c47-f550-4e24-a4ec-59f3e7fa2f30 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
122872025-04-30T06:13:32.715ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2477, tv_nsec: 513968473 } })
122882025-04-30T06:13:32.715ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
122892025-04-30T06:13:32.715ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = cd956c47-f550-4e24-a4ec-59f3e7fa2f30 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
122902025-04-30T06:13:32.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122912025-04-30T06:13:32.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
122922025-04-30T06:13:32.719ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 3d 7d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e =}..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53182 req_id = 9a1dd660-d178-4654-9b52-4727eb1f176d
122932025-04-30T06:13:32.719ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15741,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53182 req_id = 9a1dd660-d178-4654-9b52-4727eb1f176d
122942025-04-30T06:13:32.719ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
122952025-04-30T06:13:32.719ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53182 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9a1dd660-d178-4654-9b52-4727eb1f176d
122962025-04-30T06:13:32.720ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
122972025-04-30T06:13:32.720ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
122982025-04-30T06:13:32.721ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = cd956c47-f550-4e24-a4ec-59f3e7fa2f30 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
122992025-04-30T06:13:32.721ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123002025-04-30T06:13:32.721ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
123012025-04-30T06:13:32.722ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
123022025-04-30T06:13:32.722ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
123032025-04-30T06:13:32.722ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
123042025-04-30T06:13:32.722ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
123052025-04-30T06:13:32.724ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2fd82f4-d2a7-4eb7-ac62-8630f1e4f2b5", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123062025-04-30T06:13:32.724ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
123072025-04-30T06:13:32.725ZINFOomicron-dev (dropshot_internal): request completed latency_us = 12475 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = cd956c47-f550-4e24-a4ec-59f3e7fa2f30 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
123082025-04-30T06:13:32.725ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd956c47-f550-4e24-a4ec-59f3e7fa2f30", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123092025-04-30T06:13:32.726ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
123102025-04-30T06:13:32.726ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
123112025-04-30T06:13:32.726ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
123122025-04-30T06:13:32.729ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "813fe043-19bd-4eff-a31e-60a2356d399a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123132025-04-30T06:13:32.729ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
123142025-04-30T06:13:32.732ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d39ebded-456c-4938-9069-8a2f88ecea97", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123152025-04-30T06:13:32.732ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
123162025-04-30T06:13:32.737ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
123172025-04-30T06:13:32.737ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 35 26 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 5&..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:49383 req_id = 4dc6a1af-2da2-4205-a426-6f76806f80df
123182025-04-30T06:13:32.737ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13606,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49383 req_id = 4dc6a1af-2da2-4205-a426-6f76806f80df
123192025-04-30T06:13:32.738ZDEBGomicron-dev (store): query key key = _dendrite._tcp
123202025-04-30T06:13:32.738ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
123212025-04-30T06:13:32.738ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:49383 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 4dc6a1af-2da2-4205-a426-6f76806f80df
123222025-04-30T06:13:32.738ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "174aab5e-047c-4023-8fca-d4516f9b6229", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123232025-04-30T06:13:32.738ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
123242025-04-30T06:13:32.738ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2477, tv_nsec: 537557653 } })
123252025-04-30T06:13:32.738ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
123262025-04-30T06:13:32.739ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
123272025-04-30T06:13:32.739ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: e7 3b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .;..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60757 req_id = d49d1369-b1cd-455a-984d-68db249b524f
123282025-04-30T06:13:32.739ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59195,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60757 req_id = d49d1369-b1cd-455a-984d-68db249b524f
123292025-04-30T06:13:32.739ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
123302025-04-30T06:13:32.739ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60757 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d49d1369-b1cd-455a-984d-68db249b524f
123312025-04-30T06:13:32.740ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
123322025-04-30T06:13:32.753ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
123332025-04-30T06:13:32.759ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 5b 2a 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 [*..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:65294 req_id = 80f129ec-28d0-4d24-9894-092981e2105a
123342025-04-30T06:13:32.759ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23338,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65294 req_id = 80f129ec-28d0-4d24-9894-092981e2105a
123352025-04-30T06:13:32.759ZDEBGomicron-dev (store): query key key = _dendrite._tcp
123362025-04-30T06:13:32.759ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
123372025-04-30T06:13:32.759ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65294 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 80f129ec-28d0-4d24-9894-092981e2105a
123382025-04-30T06:13:32.760ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2477, tv_nsec: 559021097 } })
123392025-04-30T06:13:32.760ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
123402025-04-30T06:13:32.761ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 71 52 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e qR..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65325 req_id = 1e435269-71c2-4263-9741-e446c5e40daf
123412025-04-30T06:13:32.761ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29010,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65325 req_id = 1e435269-71c2-4263-9741-e446c5e40daf
123422025-04-30T06:13:32.761ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
123432025-04-30T06:13:32.761ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65325 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1e435269-71c2-4263-9741-e446c5e40daf
123442025-04-30T06:13:32.761ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
123452025-04-30T06:13:32.761ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85388a5c-2e23-449a-a19b-949199eaafbd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123462025-04-30T06:13:32.762ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
123472025-04-30T06:13:32.764ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a259a645-2fdb-4491-908d-1f25c5bf165d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123482025-04-30T06:13:32.764ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
123492025-04-30T06:13:32.765ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1261130-a44e-4fac-9907-ff1d0c3aa2fc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123502025-04-30T06:13:32.766ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
123512025-04-30T06:13:32.767ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a541752e-f711-4254-af3e-f9c63de9e324", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123522025-04-30T06:13:32.767ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
123532025-04-30T06:13:32.768ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "edb724b2-dae1-4854-a8fe-204fef6904c7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123542025-04-30T06:13:32.768ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
123552025-04-30T06:13:32.769ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5656b86b-df34-49af-a121-457c36621ae5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123562025-04-30T06:13:32.769ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
123572025-04-30T06:13:32.771ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f34ea536-c64e-4b14-9a81-f30195fff4c4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123582025-04-30T06:13:32.771ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
123592025-04-30T06:13:32.774ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f5d744d-eced-4a17-bd99-74219cd3db21", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123602025-04-30T06:13:32.774ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
123612025-04-30T06:13:32.776ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa382830-d6ff-4c60-84ca-1f347a10653d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123622025-04-30T06:13:32.776ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
123632025-04-30T06:13:32.780ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "48c796b0-4081-442c-806d-430db4cbf8d9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123642025-04-30T06:13:32.780ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
123652025-04-30T06:13:32.783ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "901a2182-4fcd-4d08-a176-c5c4b0ff997a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123662025-04-30T06:13:32.783ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
123672025-04-30T06:13:32.787ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4e0e5b85-b80d-4545-906e-8c8986c3ba60", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123682025-04-30T06:13:32.787ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
123692025-04-30T06:13:32.789ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed678c23-35df-4aa3-af85-5bb1e8037e43", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123702025-04-30T06:13:32.790ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
123712025-04-30T06:13:32.791ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "934038c1-2657-48bd-ad0f-87d75667cd52", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123722025-04-30T06:13:32.791ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
123732025-04-30T06:13:32.793ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5d45667-9a7b-4460-a89d-3998d25d041e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123742025-04-30T06:13:32.793ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
123752025-04-30T06:13:32.793ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "536d176e-e0e1-4fa6-88ec-9ccce255f3f3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123762025-04-30T06:13:32.793ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
123772025-04-30T06:13:32.794ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad5ac023-4a86-4f4f-a47c-8591e48ac486", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123782025-04-30T06:13:32.795ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
123792025-04-30T06:13:32.796ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cd27bff-4c0e-49c1-92d7-985faa7d205b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123802025-04-30T06:13:32.796ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
123812025-04-30T06:13:32.798ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bcab133-6931-4553-8945-7548494bbaf4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123822025-04-30T06:13:32.798ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
123832025-04-30T06:13:32.800ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d225d40-40ca-45ac-8467-870cb8ee4087", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123842025-04-30T06:13:32.801ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
123852025-04-30T06:13:32.803ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8919a68-32fa-4b1d-8c1f-21c5c489bafa", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:31 GMT"} })
123862025-04-30T06:13:32.803ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
123872025-04-30T06:13:32.804ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
123882025-04-30T06:13:32.804ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
123892025-04-30T06:13:32.804ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
123902025-04-30T06:13:32.804ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
123912025-04-30T06:13:32.805ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
123922025-04-30T06:13:32.807ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a2d408b-5d17-42cb-b49f-f2317a92d053", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123932025-04-30T06:13:32.807ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
123942025-04-30T06:13:32.809ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "07b739e1-934f-4f40-b4c3-e55175906d33", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123952025-04-30T06:13:32.809ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
123962025-04-30T06:13:32.811ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4af1b174-75b0-4015-875b-4f97a6aac8e1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123972025-04-30T06:13:32.812ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
123982025-04-30T06:13:32.813ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e7813524-827b-442c-8859-8cda3d9585cd", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
123992025-04-30T06:13:32.813ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
124002025-04-30T06:13:32.814ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "044668ef-54ac-4e20-9dfb-009aaf89c1e4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124012025-04-30T06:13:32.814ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
124022025-04-30T06:13:32.815ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43f39d69-7a07-4206-a899-23ba754629bc", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124032025-04-30T06:13:32.816ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
124042025-04-30T06:13:32.817ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3095c84e-bd5c-4d22-8b67-318d90568858", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124052025-04-30T06:13:32.817ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
124062025-04-30T06:13:32.818ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bde13968-ba8a-441b-a256-4daf42f80499", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124072025-04-30T06:13:32.818ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
124082025-04-30T06:13:32.820ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "11d1d74c-bed6-4c6e-a1cb-a31cc2a28b54", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124092025-04-30T06:13:32.820ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
124102025-04-30T06:13:32.822ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9f231bf9-1ec4-4e51-a9a0-a85de804b28e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124112025-04-30T06:13:32.822ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
124122025-04-30T06:13:32.823ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e2c82f6-ad96-43b2-81f1-55ffcf5be45c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124132025-04-30T06:13:32.823ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
124142025-04-30T06:13:32.825ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "30b7015e-da23-4458-a42e-c55083474631", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124152025-04-30T06:13:32.825ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
124162025-04-30T06:13:32.827ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "363893c3-0566-4aa7-9334-4fafba38d07e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124172025-04-30T06:13:32.827ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
124182025-04-30T06:13:32.829ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "544b4656-f935-4d07-becd-e4cfbc67a3d9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124192025-04-30T06:13:32.829ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
124202025-04-30T06:13:32.830ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b785881-dc83-4b5d-94f9-4fb74bd2f0d9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124212025-04-30T06:13:32.830ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
124222025-04-30T06:13:32.831ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "96e7fe4c-10d8-4907-b2d3-043ae4ca1494", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124232025-04-30T06:13:32.831ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
124242025-04-30T06:13:32.832ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffa3426e-93bf-46d8-b075-b50b1b7588a5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124252025-04-30T06:13:32.832ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
124262025-04-30T06:13:32.833ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "294e4c35-f879-4da4-9608-58fb5f687ffe", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124272025-04-30T06:13:32.833ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
124282025-04-30T06:13:32.833ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124292025-04-30T06:13:32.833ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124302025-04-30T06:13:32.833ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124312025-04-30T06:13:32.834ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124322025-04-30T06:13:32.835ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124332025-04-30T06:13:32.838ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
124342025-04-30T06:13:32.838ZDEBGomicron-dev (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
124352025-04-30T06:13:32.838ZDEBGomicron-dev (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
124362025-04-30T06:13:32.838ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
124372025-04-30T06:13:32.841ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
124382025-04-30T06:13:32.841ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124392025-04-30T06:13:32.841ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
124402025-04-30T06:13:32.842ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124412025-04-30T06:13:32.846ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
124422025-04-30T06:13:32.847ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124432025-04-30T06:13:32.847ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
124442025-04-30T06:13:32.847ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124452025-04-30T06:13:32.859ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124462025-04-30T06:13:32.867ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
124472025-04-30T06:13:32.867ZDEBGomicron-dev (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
124482025-04-30T06:13:32.868ZDEBGomicron-dev (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
124492025-04-30T06:13:32.868ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
124502025-04-30T06:13:32.871ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
124512025-04-30T06:13:32.872ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
124522025-04-30T06:13:32.872ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
124532025-04-30T06:13:32.872ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124542025-04-30T06:13:32.872ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124552025-04-30T06:13:32.872ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124562025-04-30T06:13:32.874ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124572025-04-30T06:13:32.875ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
124582025-04-30T06:13:32.875ZDEBGomicron-dev (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
124592025-04-30T06:13:32.875ZDEBGomicron-dev (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
124602025-04-30T06:13:32.876ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
124612025-04-30T06:13:32.881ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
124622025-04-30T06:13:32.888ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124632025-04-30T06:13:32.889ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
124642025-04-30T06:13:32.900ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
124652025-04-30T06:13:32.900ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124662025-04-30T06:13:32.900ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124672025-04-30T06:13:32.902ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124682025-04-30T06:13:32.933ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
124692025-04-30T06:13:32.933ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124702025-04-30T06:13:32.933ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124712025-04-30T06:13:32.933ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
124722025-04-30T06:13:32.935ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124732025-04-30T06:13:32.989ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
124742025-04-30T06:13:32.990ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
124752025-04-30T06:13:32.990ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:33235 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
124762025-04-30T06:13:32.991ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:33235 req_id = 1380c5c9-d1d4-45aa-b40f-1a57178bcce3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
124772025-04-30T06:13:32.992ZINFOomicron-dev (dropshot): request completed latency_us = 608 local_addr = [::1]:61881 method = GET remote_addr = [::1]:33235 req_id = 1380c5c9-d1d4-45aa-b40f-1a57178bcce3 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
124782025-04-30T06:13:32.992ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "1380c5c9-d1d4-45aa-b40f-1a57178bcce3", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
124792025-04-30T06:13:32.994ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124802025-04-30T06:13:32.995ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
124812025-04-30T06:13:32.995ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
124822025-04-30T06:13:32.995ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
124832025-04-30T06:13:32.998ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
124842025-04-30T06:13:32.998ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
124852025-04-30T06:13:32.998ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124862025-04-30T06:13:32.998ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124872025-04-30T06:13:32.999ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124882025-04-30T06:13:33.002ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
124892025-04-30T06:13:33.002ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124902025-04-30T06:13:33.003ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
124912025-04-30T06:13:33.003ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
124922025-04-30T06:13:33.003ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
124932025-04-30T06:13:33.007ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
124942025-04-30T06:13:33.007ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
124952025-04-30T06:13:33.007ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
124962025-04-30T06:13:33.008ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
124972025-04-30T06:13:33.014ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
124982025-04-30T06:13:33.014ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
124992025-04-30T06:13:33.014ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125002025-04-30T06:13:33.045ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
125012025-04-30T06:13:33.046ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
125022025-04-30T06:13:33.046ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 515.567995ms iteration = 44 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
125032025-04-30T06:13:33.046ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 45 reason = Signaled
125042025-04-30T06:13:33.046ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
125052025-04-30T06:13:33.046ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
125062025-04-30T06:13:33.046ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
125072025-04-30T06:13:33.046ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
125082025-04-30T06:13:33.047ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
125092025-04-30T06:13:33.058ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
125102025-04-30T06:13:33.061ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125112025-04-30T06:13:33.065ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
125122025-04-30T06:13:33.065ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
125132025-04-30T06:13:33.065ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
125142025-04-30T06:13:33.065ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
125152025-04-30T06:13:33.069ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
125162025-04-30T06:13:33.069ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
125172025-04-30T06:13:33.070ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: b4 73 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .s..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60456 req_id = fcd55cb4-e4e8-421c-8427-52fdb3451516
125182025-04-30T06:13:33.070ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46195,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60456 req_id = fcd55cb4-e4e8-421c-8427-52fdb3451516
125192025-04-30T06:13:33.070ZDEBGomicron-dev (store): query key key = _mgs._tcp
125202025-04-30T06:13:33.070ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
125212025-04-30T06:13:33.070ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60456 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = fcd55cb4-e4e8-421c-8427-52fdb3451516
125222025-04-30T06:13:33.071ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2477, tv_nsec: 869995834 } })
125232025-04-30T06:13:33.071ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
125242025-04-30T06:13:33.071ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d9 2f 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ./..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64078 req_id = 8e3e678e-c509-40a3-9bd2-eb899b3df5a4
125252025-04-30T06:13:33.071ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55599,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64078 req_id = 8e3e678e-c509-40a3-9bd2-eb899b3df5a4
125262025-04-30T06:13:33.071ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
125272025-04-30T06:13:33.072ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64078 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 8e3e678e-c509-40a3-9bd2-eb899b3df5a4
125282025-04-30T06:13:33.187ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
125292025-04-30T06:13:33.188ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
125302025-04-30T06:13:33.188ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
125312025-04-30T06:13:33.188ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
125322025-04-30T06:13:33.188ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 16 ee 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:56370 req_id = f411cfba-0ce5-4503-b2fa-46dd0fc2944c
125332025-04-30T06:13:33.188ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
125342025-04-30T06:13:33.188ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5870,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56370 req_id = f411cfba-0ce5-4503-b2fa-46dd0fc2944c
125352025-04-30T06:13:33.188ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
125362025-04-30T06:13:33.189ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
125372025-04-30T06:13:33.189ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5870, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56370 req_id = f411cfba-0ce5-4503-b2fa-46dd0fc2944c
125382025-04-30T06:13:33.189ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
125392025-04-30T06:13:33.189ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
125402025-04-30T06:13:33.189ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125412025-04-30T06:13:33.189ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
125422025-04-30T06:13:33.189ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
125432025-04-30T06:13:33.189ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
125442025-04-30T06:13:33.191ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125452025-04-30T06:13:33.191ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "531107bd-ea43-48c0-b5c6-9acafdcb03f8", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125462025-04-30T06:13:33.192ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
125472025-04-30T06:13:33.195ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
125482025-04-30T06:13:33.195ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
125492025-04-30T06:13:33.197ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4695caf0-3c13-43a7-9e6f-2c95610c350b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125502025-04-30T06:13:33.197ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
125512025-04-30T06:13:33.198ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
125522025-04-30T06:13:33.199ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
125532025-04-30T06:13:33.199ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
125542025-04-30T06:13:33.203ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "327ad674-1240-4831-9466-93205deae8bd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125552025-04-30T06:13:33.204ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
125562025-04-30T06:13:33.205ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
125572025-04-30T06:13:33.205ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125582025-04-30T06:13:33.205ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
125592025-04-30T06:13:33.206ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125602025-04-30T06:13:33.206ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfb90cb4-d397-4e89-b550-b718258cb6ba", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125612025-04-30T06:13:33.206ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
125622025-04-30T06:13:33.209ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae82bade-59e0-46f8-8b58-6f4c03bf6749", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125632025-04-30T06:13:33.209ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
125642025-04-30T06:13:33.209ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
125652025-04-30T06:13:33.209ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
125662025-04-30T06:13:33.210ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
125672025-04-30T06:13:33.214ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1a0b9e0-f90d-4537-87e6-74a8b800740d", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125682025-04-30T06:13:33.216ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
125692025-04-30T06:13:33.222ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e3e09b2-e974-4f9f-abcb-f2f231f35b9b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125702025-04-30T06:13:33.222ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
125712025-04-30T06:13:33.230ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ced7311-7463-4d9c-afcc-5c261e144aa3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125722025-04-30T06:13:33.230ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
125732025-04-30T06:13:33.231ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd0707a9-26d0-4ff3-a347-a645767451bf", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125742025-04-30T06:13:33.232ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
125752025-04-30T06:13:33.233ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73ff74c9-2e83-4652-b925-6449d05f3ddc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125762025-04-30T06:13:33.233ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
125772025-04-30T06:13:33.234ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3c322e1-e3fa-4d73-a635-3eb8ef543db5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125782025-04-30T06:13:33.234ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
125792025-04-30T06:13:33.238ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4da7d5e-1194-47c5-b502-9c9158debee2", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125802025-04-30T06:13:33.238ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
125812025-04-30T06:13:33.240ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1e0a7e6-8563-428b-b12d-2a514578444e", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125822025-04-30T06:13:33.240ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
125832025-04-30T06:13:33.245ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d945806b-a77e-4dfe-b1c8-1108ad84e40a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125842025-04-30T06:13:33.245ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
125852025-04-30T06:13:33.248ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a06d344f-cb96-47c0-8aa0-18a326888f53", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125862025-04-30T06:13:33.248ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
125872025-04-30T06:13:33.251ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "41c062b0-82eb-4af5-90b9-d3f49cace052", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125882025-04-30T06:13:33.251ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
125892025-04-30T06:13:33.253ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
125902025-04-30T06:13:33.255ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "461d52dc-2ea6-4ad4-9239-8257dead2206", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125912025-04-30T06:13:33.255ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
125922025-04-30T06:13:33.262ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5efe88de-5beb-46e4-bac9-b10c57437bb6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125932025-04-30T06:13:33.264ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
125942025-04-30T06:13:33.271ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1298b8e6-f553-4d5a-a61c-cbd7c0e31644", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125952025-04-30T06:13:33.272ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
125962025-04-30T06:13:33.276ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "63d2adb7-aad5-4774-bd39-d51f09f2272c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125972025-04-30T06:13:33.276ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
125982025-04-30T06:13:33.280ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "170ea5fc-7f04-4366-b26f-acca740912ea", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
125992025-04-30T06:13:33.281ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
126002025-04-30T06:13:33.282ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af2c0fff-0fca-4d50-afc5-b0fc3a1560c1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126012025-04-30T06:13:33.282ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
126022025-04-30T06:13:33.285ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b41b7b47-9009-47e6-9c9c-4a14d016d131", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126032025-04-30T06:13:33.285ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
126042025-04-30T06:13:33.286ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "37b9c23f-e140-4c93-a545-b10b091a8cab", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126052025-04-30T06:13:33.286ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
126062025-04-30T06:13:33.289ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
126072025-04-30T06:13:33.289ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "acb787b0-935d-4545-a0d7-7cd51aecfd26", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126082025-04-30T06:13:33.290ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
126092025-04-30T06:13:33.292ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f0aaff7-d53e-4228-838a-eb7296f15c1f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126102025-04-30T06:13:33.293ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
126112025-04-30T06:13:33.294ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ea22518-4d5b-4369-8d91-475d0618d3ee", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126122025-04-30T06:13:33.295ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
126132025-04-30T06:13:33.297ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "225921f4-8349-4510-9d11-8ab7e18af400", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126142025-04-30T06:13:33.297ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
126152025-04-30T06:13:33.299ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "131c2ec5-ad4b-4983-a837-c045d4dfd21d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126162025-04-30T06:13:33.299ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
126172025-04-30T06:13:33.301ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7985471a-3dfe-484c-9192-5a026e43118e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126182025-04-30T06:13:33.301ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
126192025-04-30T06:13:33.302ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6dc829b-0d56-4b86-a0a6-bcf98f77cbed", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126202025-04-30T06:13:33.302ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
126212025-04-30T06:13:33.303ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cad4b4fc-d3d5-4770-845a-88d8de72802d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126222025-04-30T06:13:33.304ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
126232025-04-30T06:13:33.304ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
126242025-04-30T06:13:33.304ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126252025-04-30T06:13:33.304ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126262025-04-30T06:13:33.304ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
126272025-04-30T06:13:33.305ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bff01965-88ea-45b8-a17b-7ed396937386", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126282025-04-30T06:13:33.305ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
126292025-04-30T06:13:33.306ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126302025-04-30T06:13:33.307ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e27b3b7-2d78-4bfd-9606-b2852990987a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126312025-04-30T06:13:33.307ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
126322025-04-30T06:13:33.309ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1388c31b-31c6-4264-86b2-6b79ba7f7392", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126332025-04-30T06:13:33.309ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
126342025-04-30T06:13:33.309ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126352025-04-30T06:13:33.312ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5679e3f0-5083-4724-999b-4f3bf8db96ba", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126362025-04-30T06:13:33.312ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
126372025-04-30T06:13:33.314ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb0b906b-8cc2-428c-844b-7ab6c653825c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126382025-04-30T06:13:33.314ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
126392025-04-30T06:13:33.316ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "09348cde-152d-44c9-9f80-9480dee67286", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126402025-04-30T06:13:33.316ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
126412025-04-30T06:13:33.318ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f50ce199-8016-483e-b7c8-fc24c9935af9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126422025-04-30T06:13:33.318ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
126432025-04-30T06:13:33.320ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
126442025-04-30T06:13:33.320ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126452025-04-30T06:13:33.320ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
126462025-04-30T06:13:33.320ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "48036b75-3a43-4029-874c-0e3be70a1f75", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126472025-04-30T06:13:33.321ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
126482025-04-30T06:13:33.321ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126492025-04-30T06:13:33.325ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126502025-04-30T06:13:33.331ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9717724a-45fd-49ec-9c58-35acfb216461", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126512025-04-30T06:13:33.331ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
126522025-04-30T06:13:33.331ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
126532025-04-30T06:13:33.331ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126542025-04-30T06:13:33.332ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b85f6458-5738-40ad-af75-07acef07efc0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126552025-04-30T06:13:33.333ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
126562025-04-30T06:13:33.335ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c618afbc-912f-4773-ac82-d15a35501be4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126572025-04-30T06:13:33.335ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
126582025-04-30T06:13:33.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "38a0baf8-c498-4823-891f-dabd4fd058a1", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126592025-04-30T06:13:33.337ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
126602025-04-30T06:13:33.338ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e4c4ed76-4a15-44c5-a277-5edab7208b61", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126612025-04-30T06:13:33.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
126622025-04-30T06:13:33.338ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126632025-04-30T06:13:33.338ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126642025-04-30T06:13:33.338ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126652025-04-30T06:13:33.339ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126662025-04-30T06:13:33.344ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126672025-04-30T06:13:33.352ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
126682025-04-30T06:13:33.352ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
126692025-04-30T06:13:33.352ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126702025-04-30T06:13:33.352ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126712025-04-30T06:13:33.352ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126722025-04-30T06:13:33.352ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126732025-04-30T06:13:33.355ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126742025-04-30T06:13:33.359ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
126752025-04-30T06:13:33.359ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
126762025-04-30T06:13:33.359ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
126772025-04-30T06:13:33.360ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
126782025-04-30T06:13:33.361ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
126792025-04-30T06:13:33.412ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
126802025-04-30T06:13:33.412ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
126812025-04-30T06:13:33.412ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126822025-04-30T06:13:33.412ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
126832025-04-30T06:13:33.413ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126842025-04-30T06:13:33.464ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
126852025-04-30T06:13:33.464ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
126862025-04-30T06:13:33.465ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:43448 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
126872025-04-30T06:13:33.465ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:43448 req_id = e3a761e1-00ca-4a12-9dcf-b8661b84b16b sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
126882025-04-30T06:13:33.466ZINFOomicron-dev (dropshot): request completed latency_us = 688 local_addr = [::1]:61881 method = GET remote_addr = [::1]:43448 req_id = e3a761e1-00ca-4a12-9dcf-b8661b84b16b response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
126892025-04-30T06:13:33.468ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
126902025-04-30T06:13:33.470ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3a761e1-00ca-4a12-9dcf-b8661b84b16b", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
126912025-04-30T06:13:33.475ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
126922025-04-30T06:13:33.475ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126932025-04-30T06:13:33.475ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
126942025-04-30T06:13:33.475ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
126952025-04-30T06:13:33.476ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
126962025-04-30T06:13:33.476ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
126972025-04-30T06:13:33.476ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
126982025-04-30T06:13:33.478ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
126992025-04-30T06:13:33.478ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
127002025-04-30T06:13:33.479ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127012025-04-30T06:13:33.480ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
127022025-04-30T06:13:33.480ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
127032025-04-30T06:13:33.480ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
127042025-04-30T06:13:33.480ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127052025-04-30T06:13:33.481ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
127062025-04-30T06:13:33.484ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127072025-04-30T06:13:33.493ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
127082025-04-30T06:13:33.494ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
127092025-04-30T06:13:33.494ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127102025-04-30T06:13:33.495ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
127112025-04-30T06:13:33.513ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
127122025-04-30T06:13:33.525ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
127132025-04-30T06:13:33.525ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
127142025-04-30T06:13:33.525ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 479.590042ms iteration = 45 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
127152025-04-30T06:13:33.525ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 46 reason = Signaled
127162025-04-30T06:13:33.525ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
127172025-04-30T06:13:33.525ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
127182025-04-30T06:13:33.525ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
127192025-04-30T06:13:33.526ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127202025-04-30T06:13:33.526ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
127212025-04-30T06:13:33.529ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127222025-04-30T06:13:33.535ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
127232025-04-30T06:13:33.535ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
127242025-04-30T06:13:33.535ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
127252025-04-30T06:13:33.535ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
127262025-04-30T06:13:33.539ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
127272025-04-30T06:13:33.539ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
127282025-04-30T06:13:33.540ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 0a bd 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:65121 req_id = c872d770-ac43-497d-9a9b-c785e91160e8
127292025-04-30T06:13:33.540ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2749,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65121 req_id = c872d770-ac43-497d-9a9b-c785e91160e8
127302025-04-30T06:13:33.541ZDEBGomicron-dev (store): query key key = _mgs._tcp
127312025-04-30T06:13:33.541ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
127322025-04-30T06:13:33.541ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65121 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c872d770-ac43-497d-9a9b-c785e91160e8
127332025-04-30T06:13:33.541ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2478, tv_nsec: 340817916 } })
127342025-04-30T06:13:33.542ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
127352025-04-30T06:13:33.542ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: bf a0 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54474 req_id = 48be83b2-8fef-4b9e-98a5-9f504864a7dd
127362025-04-30T06:13:33.542ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49056,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54474 req_id = 48be83b2-8fef-4b9e-98a5-9f504864a7dd
127372025-04-30T06:13:33.543ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
127382025-04-30T06:13:33.543ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54474 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 48be83b2-8fef-4b9e-98a5-9f504864a7dd
127392025-04-30T06:13:33.580ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
127402025-04-30T06:13:33.581ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
127412025-04-30T06:13:33.581ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
127422025-04-30T06:13:33.581ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127432025-04-30T06:13:33.581ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
127442025-04-30T06:13:33.581ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: cb 48 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .H..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54456 req_id = daf62d4f-03ff-4d7a-bdeb-c0e5d22a7df6
127452025-04-30T06:13:33.581ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52040,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54456 req_id = daf62d4f-03ff-4d7a-bdeb-c0e5d22a7df6
127462025-04-30T06:13:33.581ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
127472025-04-30T06:13:33.581ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52040, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54456 req_id = daf62d4f-03ff-4d7a-bdeb-c0e5d22a7df6
127482025-04-30T06:13:33.582ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
127492025-04-30T06:13:33.582ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
127502025-04-30T06:13:33.582ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127512025-04-30T06:13:33.582ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
127522025-04-30T06:13:33.584ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad4f73b8-58fd-4c26-9551-8040f3a4f497", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127532025-04-30T06:13:33.584ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
127542025-04-30T06:13:33.586ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c794864-59c9-414e-9004-10b76217b54c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127552025-04-30T06:13:33.586ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
127562025-04-30T06:13:33.586ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
127572025-04-30T06:13:33.593ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d23b0df6-ffb7-4c79-ba73-b037f8e49390", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127582025-04-30T06:13:33.593ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
127592025-04-30T06:13:33.595ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
127602025-04-30T06:13:33.595ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127612025-04-30T06:13:33.595ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
127622025-04-30T06:13:33.595ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127632025-04-30T06:13:33.600ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
127642025-04-30T06:13:33.600ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
127652025-04-30T06:13:33.601ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "78244dc9-95b0-4855-a2ca-c771b1d0a1ec", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127662025-04-30T06:13:33.601ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
127672025-04-30T06:13:33.603ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
127682025-04-30T06:13:33.603ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf3d8d38-c36b-4613-a38e-8ea946d933cc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127692025-04-30T06:13:33.603ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
127702025-04-30T06:13:33.607ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "032eea92-31d9-4c71-a9b2-5c34c992aed3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127712025-04-30T06:13:33.610ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
127722025-04-30T06:13:33.621ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "47373878-0272-47ad-8674-39c57cc5b628", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127732025-04-30T06:13:33.622ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
127742025-04-30T06:13:33.627ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "83e8693d-9443-40bd-ac5c-1410e13f1668", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127752025-04-30T06:13:33.627ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
127762025-04-30T06:13:33.630ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d57c22be-3174-444f-9bdb-1c3ce323fff1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127772025-04-30T06:13:33.630ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
127782025-04-30T06:13:33.632ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9f30abe-b6b4-4508-9fc8-9d897542a2f5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127792025-04-30T06:13:33.633ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
127802025-04-30T06:13:33.636ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "205cfa3d-1ddb-4098-a591-f8c68870be81", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127812025-04-30T06:13:33.637ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
127822025-04-30T06:13:33.638ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6b87372-1036-4863-81ed-54b3ac83eca9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127832025-04-30T06:13:33.639ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
127842025-04-30T06:13:33.647ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2175adfa-f3d5-4058-98ac-18526669a661", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127852025-04-30T06:13:33.648ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
127862025-04-30T06:13:33.658ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
127872025-04-30T06:13:33.659ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e88890b-ad54-4526-acbe-c0252e67156d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127882025-04-30T06:13:33.660ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
127892025-04-30T06:13:33.662ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f996bde-800b-4e90-9628-bae306796057", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127902025-04-30T06:13:33.662ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
127912025-04-30T06:13:33.668ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e30334b1-449e-4b9a-a344-e4bd8b01897c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127922025-04-30T06:13:33.668ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
127932025-04-30T06:13:33.668ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ae058c48-f57d-46bb-bb3d-3ac880150803", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127942025-04-30T06:13:33.668ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
127952025-04-30T06:13:33.668ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a003ccd-7040-4e41-bd0a-fe0cdf25357b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127962025-04-30T06:13:33.668ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
127972025-04-30T06:13:33.669ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46896e27-af2c-4e54-8559-7b7e2bcbfc3a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
127982025-04-30T06:13:33.669ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
127992025-04-30T06:13:33.670ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6612b52b-8895-4e28-b87e-8fd54ed47362", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128002025-04-30T06:13:33.670ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
128012025-04-30T06:13:33.671ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a000a8ac-a16b-4c03-ae99-b1291a5ca8f7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128022025-04-30T06:13:33.672ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
128032025-04-30T06:13:33.675ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "923c5b8e-ce13-47ee-ac4a-607ca593b865", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128042025-04-30T06:13:33.675ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
128052025-04-30T06:13:33.677ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b694dbbb-54b3-4f82-960f-804d91daa16d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128062025-04-30T06:13:33.678ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
128072025-04-30T06:13:33.682ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ee3f6ff-e35b-4056-92e4-81a399dd8dee", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128082025-04-30T06:13:33.682ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
128092025-04-30T06:13:33.686ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "826037a2-2ffa-4ecd-822e-d9f711a005ff", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128102025-04-30T06:13:33.687ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
128112025-04-30T06:13:33.689ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1058170b-e98e-4f01-9623-e437fe412538", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128122025-04-30T06:13:33.689ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
128132025-04-30T06:13:33.692ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e6c2ddf-cad6-454a-8289-a13cc493e817", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128142025-04-30T06:13:33.692ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
128152025-04-30T06:13:33.694ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "623099ac-12ef-4480-bd32-5824bf9188b5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128162025-04-30T06:13:33.694ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
128172025-04-30T06:13:33.698ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db339bbe-1b30-4353-aa0a-a990c92588e4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128182025-04-30T06:13:33.698ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
128192025-04-30T06:13:33.701ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e230ce0e-98e2-4aba-baf2-5f9853a890e5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128202025-04-30T06:13:33.701ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
128212025-04-30T06:13:33.701ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ad51849-4ade-421e-b0a4-17c1db382113", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128222025-04-30T06:13:33.701ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
128232025-04-30T06:13:33.702ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0964110e-c48f-4f55-97a2-62631e0d3856", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128242025-04-30T06:13:33.703ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
128252025-04-30T06:13:33.705ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b006846-ac31-4eff-b4a0-8e843a50cee3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128262025-04-30T06:13:33.705ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
128272025-04-30T06:13:33.707ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca8107f9-6912-4464-8ec6-b6745a5347fe", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128282025-04-30T06:13:33.707ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
128292025-04-30T06:13:33.708ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "957e907b-e4e5-4268-b9e6-1a1453ba57cb", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128302025-04-30T06:13:33.709ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
128312025-04-30T06:13:33.709ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
128322025-04-30T06:13:33.709ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128332025-04-30T06:13:33.709ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128342025-04-30T06:13:33.709ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
128352025-04-30T06:13:33.711ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128362025-04-30T06:13:33.714ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128372025-04-30T06:13:33.716ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6644755b-483c-4bc6-9948-4a0058242f36", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:32 GMT"} })
128382025-04-30T06:13:33.716ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
128392025-04-30T06:13:33.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
128402025-04-30T06:13:33.720ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
128412025-04-30T06:13:33.720ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
128422025-04-30T06:13:33.720ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128432025-04-30T06:13:33.720ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
128442025-04-30T06:13:33.722ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128452025-04-30T06:13:33.724ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a3a17498-8155-480d-b45d-66cccf90222d", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128462025-04-30T06:13:33.725ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
128472025-04-30T06:13:33.729ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128482025-04-30T06:13:33.729ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
128492025-04-30T06:13:33.729ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5c92f97-22de-4607-9a33-ab186ae4aa9d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128502025-04-30T06:13:33.729ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
128512025-04-30T06:13:33.729ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128522025-04-30T06:13:33.729ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b864a1fc-5eff-4f4f-9e58-ebb069e7a736", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128532025-04-30T06:13:33.729ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
128542025-04-30T06:13:33.729ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fc73167-4000-4549-a073-eb0e79e9a0b6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128552025-04-30T06:13:33.730ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
128562025-04-30T06:13:33.731ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8182656e-0edf-4640-8040-87e0a543499f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128572025-04-30T06:13:33.732ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
128582025-04-30T06:13:33.732ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cca08ea5-73df-416a-966e-b0e3d41a9009", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128592025-04-30T06:13:33.733ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
128602025-04-30T06:13:33.735ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
128612025-04-30T06:13:33.735ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
128622025-04-30T06:13:33.735ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
128632025-04-30T06:13:33.735ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
128642025-04-30T06:13:33.735ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "358ec065-27ed-4f93-bf33-f11bfce5505e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128652025-04-30T06:13:33.735ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
128662025-04-30T06:13:33.736ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "36bd50b8-81ef-4bf9-ae05-d3390c22a4b4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128672025-04-30T06:13:33.736ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
128682025-04-30T06:13:33.737ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81d3599e-fe25-4b4f-8a83-2b702192393f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128692025-04-30T06:13:33.737ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
128702025-04-30T06:13:33.738ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128712025-04-30T06:13:33.738ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
128722025-04-30T06:13:33.738ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128732025-04-30T06:13:33.740ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
128742025-04-30T06:13:33.742ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128752025-04-30T06:13:33.754ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
128762025-04-30T06:13:33.754ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
128772025-04-30T06:13:33.754ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128782025-04-30T06:13:33.755ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
128792025-04-30T06:13:33.755ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128802025-04-30T06:13:33.755ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
128812025-04-30T06:13:33.758ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128822025-04-30T06:13:33.765ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
128832025-04-30T06:13:33.765ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
128842025-04-30T06:13:33.765ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
128852025-04-30T06:13:33.769ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
128862025-04-30T06:13:33.796ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
128872025-04-30T06:13:33.848ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
128882025-04-30T06:13:33.848ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
128892025-04-30T06:13:33.848ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
128902025-04-30T06:13:33.848ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
128912025-04-30T06:13:33.849ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
128922025-04-30T06:13:33.854ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
128932025-04-30T06:13:33.854ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
128942025-04-30T06:13:33.855ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:40692 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
128952025-04-30T06:13:33.856ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:40692 req_id = edeaa44d-a310-40dd-9ce1-f096e9ea496d sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
128962025-04-30T06:13:33.856ZINFOomicron-dev (dropshot): request completed latency_us = 592 local_addr = [::1]:61881 method = GET remote_addr = [::1]:40692 req_id = edeaa44d-a310-40dd-9ce1-f096e9ea496d response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
128972025-04-30T06:13:33.857ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "edeaa44d-a310-40dd-9ce1-f096e9ea496d", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
128982025-04-30T06:13:33.860ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
128992025-04-30T06:13:33.862ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
129002025-04-30T06:13:33.862ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
129012025-04-30T06:13:33.862ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
129022025-04-30T06:13:33.865ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
129032025-04-30T06:13:33.865ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129042025-04-30T06:13:33.865ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129052025-04-30T06:13:33.865ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129062025-04-30T06:13:33.866ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129072025-04-30T06:13:33.869ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129082025-04-30T06:13:33.871ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
129092025-04-30T06:13:33.871ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129102025-04-30T06:13:33.871ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
129112025-04-30T06:13:33.873ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129122025-04-30T06:13:33.875ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
129132025-04-30T06:13:33.875ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129142025-04-30T06:13:33.876ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129152025-04-30T06:13:33.877ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129162025-04-30T06:13:33.880ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129172025-04-30T06:13:33.880ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
129182025-04-30T06:13:33.884ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129192025-04-30T06:13:33.911ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
129202025-04-30T06:13:33.912ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
129212025-04-30T06:13:33.912ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 386.400948ms iteration = 46 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
129222025-04-30T06:13:33.912ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 47 reason = Signaled
129232025-04-30T06:13:33.912ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
129242025-04-30T06:13:33.912ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129252025-04-30T06:13:33.912ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129262025-04-30T06:13:33.912ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129272025-04-30T06:13:33.913ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129282025-04-30T06:13:33.918ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129292025-04-30T06:13:33.925ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
129302025-04-30T06:13:33.925ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
129312025-04-30T06:13:33.925ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
129322025-04-30T06:13:33.926ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
129332025-04-30T06:13:33.944ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
129342025-04-30T06:13:33.944ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
129352025-04-30T06:13:33.945ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 92 f8 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:65323 req_id = 288619d2-a899-4e96-8605-2d0a64827138
129362025-04-30T06:13:33.946ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 37624,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65323 req_id = 288619d2-a899-4e96-8605-2d0a64827138
129372025-04-30T06:13:33.946ZDEBGomicron-dev (store): query key key = _mgs._tcp
129382025-04-30T06:13:33.947ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
129392025-04-30T06:13:33.947ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:65323 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 288619d2-a899-4e96-8605-2d0a64827138
129402025-04-30T06:13:33.948ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2478, tv_nsec: 747453801 } })
129412025-04-30T06:13:33.948ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
129422025-04-30T06:13:33.950ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a9 f4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59451 req_id = d651352e-4a8c-46c5-938b-72f724e82f30
129432025-04-30T06:13:33.950ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 43508,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59451 req_id = d651352e-4a8c-46c5-938b-72f724e82f30
129442025-04-30T06:13:33.951ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
129452025-04-30T06:13:33.951ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59451 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d651352e-4a8c-46c5-938b-72f724e82f30
129462025-04-30T06:13:33.952ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
129472025-04-30T06:13:34.004ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
129482025-04-30T06:13:34.004ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
129492025-04-30T06:13:34.004ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129502025-04-30T06:13:34.004ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
129512025-04-30T06:13:34.006ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129522025-04-30T06:13:34.020ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
129532025-04-30T06:13:34.028ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
129542025-04-30T06:13:34.030ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ac 9f 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:63679 req_id = 6c28ba62-db35-45f8-a643-7c4d2dcc4632
129552025-04-30T06:13:34.031ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
129562025-04-30T06:13:34.031ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129572025-04-30T06:13:34.031ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
129582025-04-30T06:13:34.031ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44191,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63679 req_id = 6c28ba62-db35-45f8-a643-7c4d2dcc4632
129592025-04-30T06:13:34.031ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
129602025-04-30T06:13:34.031ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 44191, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:63679 req_id = 6c28ba62-db35-45f8-a643-7c4d2dcc4632
129612025-04-30T06:13:34.031ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129622025-04-30T06:13:34.031ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
129632025-04-30T06:13:34.031ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
129642025-04-30T06:13:34.031ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
129652025-04-30T06:13:34.034ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cac803c-fe25-4186-92cb-1d790ed3950f", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
129662025-04-30T06:13:34.035ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
129672025-04-30T06:13:34.035ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
129682025-04-30T06:13:34.035ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
129692025-04-30T06:13:34.036ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
129702025-04-30T06:13:34.037ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "378ff4ff-80eb-4ad5-90a8-ae15f76ed2de", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
129712025-04-30T06:13:34.037ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
129722025-04-30T06:13:34.040ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6619af9b-bd83-4c5c-afd2-48965e54dbd7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129732025-04-30T06:13:34.040ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
129742025-04-30T06:13:34.042ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2dc0a3dd-c56f-4329-966b-65a85de84ff0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129752025-04-30T06:13:34.042ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
129762025-04-30T06:13:34.044ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7d1436f6-a7a5-4f7a-9aec-eda97db15a94", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129772025-04-30T06:13:34.044ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
129782025-04-30T06:13:34.046ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecfa6624-cdf7-4e90-97fe-cb4620722b41", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129792025-04-30T06:13:34.046ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
129802025-04-30T06:13:34.048ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5337a4f-1232-48bf-baea-203a70d6ca06", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129812025-04-30T06:13:34.048ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
129822025-04-30T06:13:34.051ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "048b4b4b-f0bc-47f9-9fd1-f40496d3d0b9", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129832025-04-30T06:13:34.051ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
129842025-04-30T06:13:34.052ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55e8e57b-70d1-410c-a831-6d70ec5ab0b4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129852025-04-30T06:13:34.052ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
129862025-04-30T06:13:34.053ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e669a556-6f7f-411a-b0ab-bb874ce6ce53", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129872025-04-30T06:13:34.053ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
129882025-04-30T06:13:34.055ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec85d74d-00fb-4fd8-9f5f-7a2aa226c463", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129892025-04-30T06:13:34.055ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
129902025-04-30T06:13:34.055ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "efa2b1c0-6a92-4afc-91fd-010c3a52455b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129912025-04-30T06:13:34.055ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
129922025-04-30T06:13:34.057ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "67a27fdc-dc18-48d0-bfe9-94fbcc08d10b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
129932025-04-30T06:13:34.057ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
129942025-04-30T06:13:34.059ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7ad68c6-6680-42d7-a0b9-7ca73b6a577c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
129952025-04-30T06:13:34.059ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
129962025-04-30T06:13:34.061ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b137365a-8352-4389-90a8-3a01f774b1dd", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
129972025-04-30T06:13:34.062ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
129982025-04-30T06:13:34.065ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6033dbe5-5611-4814-a276-5134eaaab9cd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
129992025-04-30T06:13:34.065ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
130002025-04-30T06:13:34.067ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "73f970ac-1795-4def-9271-143492c7b7b4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130012025-04-30T06:13:34.068ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
130022025-04-30T06:13:34.070ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd3f8a85-6bcc-4bf5-804d-f5e0a90cc824", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130032025-04-30T06:13:34.070ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
130042025-04-30T06:13:34.072ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c44bcf4b-aedc-4272-b2db-3842980f0f08", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130052025-04-30T06:13:34.072ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
130062025-04-30T06:13:34.074ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5924e2e-3068-4a27-b0d2-5efc8d83177b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130072025-04-30T06:13:34.074ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
130082025-04-30T06:13:34.075ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c193e467-22db-4d7a-aae7-7d1ab4c1d2e3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130092025-04-30T06:13:34.075ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
130102025-04-30T06:13:34.078ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b293efae-058e-469f-b72f-d72988854ce7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130112025-04-30T06:13:34.078ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
130122025-04-30T06:13:34.082ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "83012ae0-dca0-47ec-a070-3192db1a27a9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130132025-04-30T06:13:34.082ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
130142025-04-30T06:13:34.084ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd1a2771-24d5-4752-a252-76505405961e", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130152025-04-30T06:13:34.084ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
130162025-04-30T06:13:34.086ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fe176b9-b93d-4f77-a4bb-07a8fbb9c494", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130172025-04-30T06:13:34.087ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
130182025-04-30T06:13:34.088ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c440a69b-4ca8-437d-b9ae-e26a6e7d61d5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130192025-04-30T06:13:34.088ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
130202025-04-30T06:13:34.088ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
130212025-04-30T06:13:34.090ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
130222025-04-30T06:13:34.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
130232025-04-30T06:13:34.090ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
130242025-04-30T06:13:34.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
130252025-04-30T06:13:34.093ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "887271c0-d7a6-4828-ac59-c16b561c80e4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130262025-04-30T06:13:34.093ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
130272025-04-30T06:13:34.094ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
130282025-04-30T06:13:34.094ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
130292025-04-30T06:13:34.096ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "beeadaaa-84f7-4625-9fcb-d4a1d53f7ba6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130302025-04-30T06:13:34.096ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
130312025-04-30T06:13:34.101ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b6d9769-9395-4411-acb4-a5033296ba5e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130322025-04-30T06:13:34.101ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
130332025-04-30T06:13:34.102ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
130342025-04-30T06:13:34.102ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
130352025-04-30T06:13:34.104ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3de6957b-bcb9-4753-bf0c-8231c687886c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130362025-04-30T06:13:34.104ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
130372025-04-30T06:13:34.105ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef6ad98b-7fa5-439d-89bf-09933678c87e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130382025-04-30T06:13:34.106ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
130392025-04-30T06:13:34.108ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6e4921b-c1d9-44a2-a5a2-6f74c8f2dafe", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130402025-04-30T06:13:34.116ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
130412025-04-30T06:13:34.116ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4daf2536-4a08-4367-a959-6ccbd1d79738", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130422025-04-30T06:13:34.116ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
130432025-04-30T06:13:34.116ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "51b85386-73d5-4b9a-8a90-d93d35d1f3a1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130442025-04-30T06:13:34.116ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
130452025-04-30T06:13:34.116ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5ec2629-c8fa-42b7-af8e-b8bdc7151e98", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130462025-04-30T06:13:34.116ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
130472025-04-30T06:13:34.117ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef294193-876e-4e90-a556-cd13433c1c9e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130482025-04-30T06:13:34.117ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
130492025-04-30T06:13:34.123ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8af4bf1-d0c5-4e5d-8cd6-3ab2ffda86e1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130502025-04-30T06:13:34.123ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
130512025-04-30T06:13:34.127ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "519ceb28-27e8-42f0-a422-5172d936d597", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130522025-04-30T06:13:34.129ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
130532025-04-30T06:13:34.138ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "55180ceb-a27a-467a-a0b0-091d37021b2d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130542025-04-30T06:13:34.138ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
130552025-04-30T06:13:34.138ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9758116-b931-4b38-9dae-1f4e4c16ecf6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130562025-04-30T06:13:34.138ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
130572025-04-30T06:13:34.144ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
130582025-04-30T06:13:34.144ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
130592025-04-30T06:13:34.144ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130602025-04-30T06:13:34.144ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
130612025-04-30T06:13:34.145ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130622025-04-30T06:13:34.145ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e0ca553-5414-4099-8f4a-4d21ae58beea", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130632025-04-30T06:13:34.145ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
130642025-04-30T06:13:34.155ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f6d826e-c7c5-4ef0-8753-3b1cec3fc4ea", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
130652025-04-30T06:13:34.155ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
130662025-04-30T06:13:34.155ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f3f954b3-a735-422d-a5e2-d7f35f9888fd", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130672025-04-30T06:13:34.155ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130682025-04-30T06:13:34.155ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
130692025-04-30T06:13:34.155ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a729b84-aad8-495f-9186-8b10d7f0604e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130702025-04-30T06:13:34.155ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
130712025-04-30T06:13:34.155ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cef6d93-efd3-47db-9d1f-739f15394ba5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
130722025-04-30T06:13:34.155ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
130732025-04-30T06:13:34.155ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
130742025-04-30T06:13:34.156ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
130752025-04-30T06:13:34.156ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
130762025-04-30T06:13:34.157ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
130772025-04-30T06:13:34.157ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130782025-04-30T06:13:34.157ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
130792025-04-30T06:13:34.159ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130802025-04-30T06:13:34.161ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
130812025-04-30T06:13:34.164ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
130822025-04-30T06:13:34.164ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
130832025-04-30T06:13:34.166ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130842025-04-30T06:13:34.166ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
130852025-04-30T06:13:34.172ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
130862025-04-30T06:13:34.173ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
130872025-04-30T06:13:34.173ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
130882025-04-30T06:13:34.173ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
130892025-04-30T06:13:34.173ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
130902025-04-30T06:13:34.173ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
130912025-04-30T06:13:34.189ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
130922025-04-30T06:13:34.191ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
130932025-04-30T06:13:34.199ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
130942025-04-30T06:13:34.200ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
130952025-04-30T06:13:34.200ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
130962025-04-30T06:13:34.200ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
130972025-04-30T06:13:34.225ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
130982025-04-30T06:13:34.276ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
130992025-04-30T06:13:34.277ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131002025-04-30T06:13:34.277ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131012025-04-30T06:13:34.277ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
131022025-04-30T06:13:34.278ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131032025-04-30T06:13:34.298ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
131042025-04-30T06:13:34.299ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
131052025-04-30T06:13:34.299ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:43328 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
131062025-04-30T06:13:34.301ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:43328 req_id = 4dadb74d-6376-4378-afc0-222e734621e6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
131072025-04-30T06:13:34.302ZINFOomicron-dev (dropshot): request completed latency_us = 556 local_addr = [::1]:61881 method = GET remote_addr = [::1]:43328 req_id = 4dadb74d-6376-4378-afc0-222e734621e6 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
131082025-04-30T06:13:34.303ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "4dadb74d-6376-4378-afc0-222e734621e6", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
131092025-04-30T06:13:34.305ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131102025-04-30T06:13:34.306ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
131112025-04-30T06:13:34.306ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
131122025-04-30T06:13:34.306ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
131132025-04-30T06:13:34.308ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
131142025-04-30T06:13:34.308ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131152025-04-30T06:13:34.308ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131162025-04-30T06:13:34.308ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131172025-04-30T06:13:34.309ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131182025-04-30T06:13:34.311ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131192025-04-30T06:13:34.315ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
131202025-04-30T06:13:34.315ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131212025-04-30T06:13:34.315ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131222025-04-30T06:13:34.315ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131232025-04-30T06:13:34.317ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
131242025-04-30T06:13:34.317ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131252025-04-30T06:13:34.317ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
131262025-04-30T06:13:34.318ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131272025-04-30T06:13:34.321ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131282025-04-30T06:13:34.322ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
131292025-04-30T06:13:34.322ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131302025-04-30T06:13:34.359ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
131312025-04-30T06:13:34.368ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
131322025-04-30T06:13:34.368ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
131332025-04-30T06:13:34.368ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 456.438915ms iteration = 47 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
131342025-04-30T06:13:34.368ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 48 reason = Signaled
131352025-04-30T06:13:34.368ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
131362025-04-30T06:13:34.369ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131372025-04-30T06:13:34.369ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131382025-04-30T06:13:34.369ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131392025-04-30T06:13:34.372ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131402025-04-30T06:13:34.379ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131412025-04-30T06:13:34.388ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
131422025-04-30T06:13:34.388ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
131432025-04-30T06:13:34.389ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
131442025-04-30T06:13:34.390ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
131452025-04-30T06:13:34.397ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
131462025-04-30T06:13:34.397ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
131472025-04-30T06:13:34.399ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: dd f8 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55360 req_id = d49802c0-869d-4c0e-9abc-6ac9825ad409
131482025-04-30T06:13:34.400ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56824,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55360 req_id = d49802c0-869d-4c0e-9abc-6ac9825ad409
131492025-04-30T06:13:34.415ZDEBGomicron-dev (store): query key key = _mgs._tcp
131502025-04-30T06:13:34.415ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
131512025-04-30T06:13:34.415ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55360 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d49802c0-869d-4c0e-9abc-6ac9825ad409
131522025-04-30T06:13:34.415ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2479, tv_nsec: 205301934 } })
131532025-04-30T06:13:34.416ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
131542025-04-30T06:13:34.416ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 0a 9f 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51059 req_id = 22cb92a9-2e58-4fab-9fb0-e8a8fd5bd9fa
131552025-04-30T06:13:34.416ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 2719,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51059 req_id = 22cb92a9-2e58-4fab-9fb0-e8a8fd5bd9fa
131562025-04-30T06:13:34.416ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
131572025-04-30T06:13:34.416ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51059 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 22cb92a9-2e58-4fab-9fb0-e8a8fd5bd9fa
131582025-04-30T06:13:34.438ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
131592025-04-30T06:13:34.438ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
131602025-04-30T06:13:34.438ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131612025-04-30T06:13:34.438ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
131622025-04-30T06:13:34.440ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131632025-04-30T06:13:34.455ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
131642025-04-30T06:13:34.470ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
131652025-04-30T06:13:34.470ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131662025-04-30T06:13:34.470ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
131672025-04-30T06:13:34.470ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131682025-04-30T06:13:34.475ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
131692025-04-30T06:13:34.475ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
131702025-04-30T06:13:34.476ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
131712025-04-30T06:13:34.504ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
131722025-04-30T06:13:34.506ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: ce 73 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .s..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54110 req_id = 8836d2df-2d61-4868-8cfc-4e28bca444a4
131732025-04-30T06:13:34.506ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52851,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54110 req_id = 8836d2df-2d61-4868-8cfc-4e28bca444a4
131742025-04-30T06:13:34.506ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
131752025-04-30T06:13:34.506ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 52851, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54110 req_id = 8836d2df-2d61-4868-8cfc-4e28bca444a4
131762025-04-30T06:13:34.506ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
131772025-04-30T06:13:34.506ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
131782025-04-30T06:13:34.506ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
131792025-04-30T06:13:34.508ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a5f4ae1-c1ed-46d6-ad5b-68d78388cf25", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
131802025-04-30T06:13:34.508ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
131812025-04-30T06:13:34.510ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9d401d2-e886-46aa-b5a3-65cde5b0c7ae", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
131822025-04-30T06:13:34.510ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
131832025-04-30T06:13:34.512ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8526a0e-3239-48cb-9ed6-568fb02c911b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
131842025-04-30T06:13:34.512ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
131852025-04-30T06:13:34.516ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b2ef3d7-a189-4230-b8aa-821ef616d1a7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
131862025-04-30T06:13:34.516ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
131872025-04-30T06:13:34.518ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
131882025-04-30T06:13:34.519ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4387713a-c401-4489-ae98-3c7ddf79265e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
131892025-04-30T06:13:34.520ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
131902025-04-30T06:13:34.526ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "432d4c22-cfa8-4c2f-a06f-f5bbfdc00744", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
131912025-04-30T06:13:34.526ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
131922025-04-30T06:13:34.530ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "db733bde-3899-4edf-a582-7386c98b82af", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
131932025-04-30T06:13:34.531ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
131942025-04-30T06:13:34.535ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "da9551b7-56a7-405f-b676-eab98257ee4b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
131952025-04-30T06:13:34.535ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
131962025-04-30T06:13:34.537ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14b28907-6b52-476b-aa8b-79013498c44c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
131972025-04-30T06:13:34.537ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
131982025-04-30T06:13:34.538ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "54cd752c-2ead-4ebb-a92a-62ca1a384df5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
131992025-04-30T06:13:34.539ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
132002025-04-30T06:13:34.540ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b645030c-7f0e-4c46-b723-305c8b46fded", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132012025-04-30T06:13:34.540ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
132022025-04-30T06:13:34.541ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29bf53ab-70e0-451f-97e4-27440505fd3e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132032025-04-30T06:13:34.541ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
132042025-04-30T06:13:34.542ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "74367298-5990-455d-a5db-33daa0bb9c78", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132052025-04-30T06:13:34.542ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
132062025-04-30T06:13:34.544ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c3808a24-eef7-4d71-b1fb-92a3651e39d4", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132072025-04-30T06:13:34.544ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
132082025-04-30T06:13:34.546ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "305a88db-836c-4828-91d3-5b60e8861c94", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132092025-04-30T06:13:34.546ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
132102025-04-30T06:13:34.548ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "85459366-b921-493e-932e-c86410b9e37e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132112025-04-30T06:13:34.548ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
132122025-04-30T06:13:34.550ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0732eaaa-5201-41b9-90db-6ddc044edb77", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132132025-04-30T06:13:34.550ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
132142025-04-30T06:13:34.551ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d498532d-ae7a-48bd-afb5-79ba2ce02112", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132152025-04-30T06:13:34.551ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
132162025-04-30T06:13:34.553ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6701fec-622a-4c37-9bb4-ac1a3a3da4a7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132172025-04-30T06:13:34.553ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
132182025-04-30T06:13:34.554ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8dd364b5-286e-4535-bcc2-35d029efe335", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132192025-04-30T06:13:34.554ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
132202025-04-30T06:13:34.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "952fa670-f607-40a2-a0c5-a229038c14e1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132212025-04-30T06:13:34.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
132222025-04-30T06:13:34.559ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9058cd9a-507d-4a35-855e-2343db800c93", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132232025-04-30T06:13:34.559ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
132242025-04-30T06:13:34.562ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb425432-4e80-4c3c-a316-09e67d6a85a8", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132252025-04-30T06:13:34.562ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
132262025-04-30T06:13:34.565ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0271724a-1c80-4214-89d2-608858b74ca9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132272025-04-30T06:13:34.565ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
132282025-04-30T06:13:34.567ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e60e143d-20fb-46e2-943a-0a85c30a4565", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132292025-04-30T06:13:34.567ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
132302025-04-30T06:13:34.569ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
132312025-04-30T06:13:34.569ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
132322025-04-30T06:13:34.569ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132332025-04-30T06:13:34.569ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
132342025-04-30T06:13:34.570ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132352025-04-30T06:13:34.573ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132362025-04-30T06:13:34.573ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "265ef52d-29ed-4c25-b015-97960b79e43c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132372025-04-30T06:13:34.573ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
132382025-04-30T06:13:34.576ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "82c29491-3e25-44c7-9c5b-19192d84b45d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132392025-04-30T06:13:34.576ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
132402025-04-30T06:13:34.579ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b576e2c8-201c-4a95-b192-0371eb42c930", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132412025-04-30T06:13:34.579ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
132422025-04-30T06:13:34.579ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
132432025-04-30T06:13:34.579ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132442025-04-30T06:13:34.579ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
132452025-04-30T06:13:34.580ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132462025-04-30T06:13:34.582ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3469189-0483-4008-9e50-39a0ea59f991", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132472025-04-30T06:13:34.582ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
132482025-04-30T06:13:34.583ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
132492025-04-30T06:13:34.583ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
132502025-04-30T06:13:34.584ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
132512025-04-30T06:13:34.587ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9aa69891-a67b-4381-9ed7-920d5591a7dc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132522025-04-30T06:13:34.587ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
132532025-04-30T06:13:34.593ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "57af7076-d5af-4b2f-a3c8-e9bb2a61713e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132542025-04-30T06:13:34.593ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
132552025-04-30T06:13:34.594ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9502e4a4-f7e5-45be-9393-7f8e3fb7f807", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132562025-04-30T06:13:34.594ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
132572025-04-30T06:13:34.596ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "15014383-de15-474f-a2da-5ae1b7fb5b48", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132582025-04-30T06:13:34.597ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
132592025-04-30T06:13:34.598ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc4c3c90-3316-4dfc-8843-e8241583a597", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132602025-04-30T06:13:34.598ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
132612025-04-30T06:13:34.600ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a63a2cd-85c2-4cb8-b0b2-5353c4314016", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132622025-04-30T06:13:34.600ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
132632025-04-30T06:13:34.602ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca106821-b68b-4b19-a130-443917bc81e4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132642025-04-30T06:13:34.602ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
132652025-04-30T06:13:34.605ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "26d8d40d-2249-495c-a25c-1068707bc920", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132662025-04-30T06:13:34.605ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
132672025-04-30T06:13:34.607ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1842abe2-0fb1-45a3-933b-8c5bf0c26c23", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132682025-04-30T06:13:34.607ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
132692025-04-30T06:13:34.609ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "243def2b-9643-4754-a295-518c5c2c22f4", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132702025-04-30T06:13:34.609ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
132712025-04-30T06:13:34.611ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "559a3c3c-688f-4351-ab2e-b9334f0611d6", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132722025-04-30T06:13:34.611ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
132732025-04-30T06:13:34.613ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a5d09ae-71c5-4df3-bd12-3629fc4165a8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132742025-04-30T06:13:34.613ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
132752025-04-30T06:13:34.614ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff23a930-36ba-45e4-80cf-752cb892d506", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132762025-04-30T06:13:34.614ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
132772025-04-30T06:13:34.615ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8f8187b-f05f-44c3-8133-26d076249dbe", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132782025-04-30T06:13:34.616ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
132792025-04-30T06:13:34.617ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0fe50d1-8f2f-4a39-9b09-7d3cb0f3fd49", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
132802025-04-30T06:13:34.617ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
132812025-04-30T06:13:34.619ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd785556-de79-4b00-9fb8-3f940d925b2e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
132822025-04-30T06:13:34.619ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
132832025-04-30T06:13:34.619ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
132842025-04-30T06:13:34.619ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
132852025-04-30T06:13:34.619ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
132862025-04-30T06:13:34.620ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
132872025-04-30T06:13:34.623ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132882025-04-30T06:13:34.627ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
132892025-04-30T06:13:34.627ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
132902025-04-30T06:13:34.627ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
132912025-04-30T06:13:34.627ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
132922025-04-30T06:13:34.627ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
132932025-04-30T06:13:34.628ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
132942025-04-30T06:13:34.629ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
132952025-04-30T06:13:34.630ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
132962025-04-30T06:13:34.634ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
132972025-04-30T06:13:34.634ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
132982025-04-30T06:13:34.634ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
132992025-04-30T06:13:34.634ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133002025-04-30T06:13:34.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
133012025-04-30T06:13:34.681ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
133022025-04-30T06:13:34.681ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bbed3c16-9fc1-4089-955c-0d2757e9f357 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
133032025-04-30T06:13:34.682ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bbed3c16-9fc1-4089-955c-0d2757e9f357 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
133042025-04-30T06:13:34.682ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bbed3c16-9fc1-4089-955c-0d2757e9f357 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
133052025-04-30T06:13:34.685ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bbed3c16-9fc1-4089-955c-0d2757e9f357 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
133062025-04-30T06:13:34.687ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
133072025-04-30T06:13:34.687ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133082025-04-30T06:13:34.687ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133092025-04-30T06:13:34.687ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
133102025-04-30T06:13:34.689ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133112025-04-30T06:13:34.689ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
133122025-04-30T06:13:34.689ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
133132025-04-30T06:13:34.689ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:64785 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
133142025-04-30T06:13:34.690ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:64785 req_id = 64cd3f04-0bea-459f-ba22-d7aae346d7d9 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
133152025-04-30T06:13:34.690ZINFOomicron-dev (dropshot): request completed latency_us = 547 local_addr = [::1]:61881 method = GET remote_addr = [::1]:64785 req_id = 64cd3f04-0bea-459f-ba22-d7aae346d7d9 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
133162025-04-30T06:13:34.691ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "64cd3f04-0bea-459f-ba22-d7aae346d7d9", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
133172025-04-30T06:13:34.695ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
133182025-04-30T06:13:34.695ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
133192025-04-30T06:13:34.695ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
133202025-04-30T06:13:34.696ZINFOomicron-dev (dropshot_internal): request completed latency_us = 14105 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bbed3c16-9fc1-4089-955c-0d2757e9f357 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
133212025-04-30T06:13:34.696ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbed3c16-9fc1-4089-955c-0d2757e9f357", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
133222025-04-30T06:13:34.697ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
133232025-04-30T06:13:34.697ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3c81ff68-daa5-4524-ab43-8d506c7a731e uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
133242025-04-30T06:13:34.698ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133252025-04-30T06:13:34.698ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3c81ff68-daa5-4524-ab43-8d506c7a731e resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
133262025-04-30T06:13:34.698ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3c81ff68-daa5-4524-ab43-8d506c7a731e roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
133272025-04-30T06:13:34.699ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3c81ff68-daa5-4524-ab43-8d506c7a731e resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
133282025-04-30T06:13:34.699ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
133292025-04-30T06:13:34.699ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133302025-04-30T06:13:34.699ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133312025-04-30T06:13:34.700ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133322025-04-30T06:13:34.701ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133332025-04-30T06:13:34.711ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
133342025-04-30T06:13:34.711ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133352025-04-30T06:13:34.711ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
133362025-04-30T06:13:34.712ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133372025-04-30T06:13:34.715ZINFOomicron-dev (dropshot_internal): request completed latency_us = 17389 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 3c81ff68-daa5-4524-ab43-8d506c7a731e response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
133382025-04-30T06:13:34.716ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "3c81ff68-daa5-4524-ab43-8d506c7a731e", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
133392025-04-30T06:13:34.716ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
133402025-04-30T06:13:34.716ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
133412025-04-30T06:13:34.716ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
133422025-04-30T06:13:34.716ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133432025-04-30T06:13:34.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
133442025-04-30T06:13:34.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
133452025-04-30T06:13:34.721ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
133462025-04-30T06:13:34.721ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
133472025-04-30T06:13:34.721ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
133482025-04-30T06:13:34.721ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:50290 method = GET remote_addr = [::1]:62154 req_id = 831105aa-6c1a-40f5-8b2f-4d312e2019b4 uri = /e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
133492025-04-30T06:13:34.722ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
133502025-04-30T06:13:34.723ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
133512025-04-30T06:13:34.723ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
133522025-04-30T06:13:34.729ZINFOomicron-dev (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 7311 local_addr = [::1]:50290 method = GET remote_addr = [::1]:62154 req_id = 831105aa-6c1a-40f5-8b2f-4d312e2019b4 response_code = 200 uri = /e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
133532025-04-30T06:13:34.729ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
133542025-04-30T06:13:34.730ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
133552025-04-30T06:13:34.730ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133562025-04-30T06:13:34.730ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133572025-04-30T06:13:34.730ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
133582025-04-30T06:13:34.731ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133592025-04-30T06:13:34.732ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133602025-04-30T06:13:34.736ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
133612025-04-30T06:13:34.743ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 5 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
133622025-04-30T06:13:34.743ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
133632025-04-30T06:13:34.743ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
133642025-04-30T06:13:34.771ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
133652025-04-30T06:13:34.771ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
133662025-04-30T06:13:34.771ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 402.538451ms iteration = 48 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
133672025-04-30T06:13:34.771ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 49 reason = Signaled
133682025-04-30T06:13:34.771ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
133692025-04-30T06:13:34.771ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133702025-04-30T06:13:34.771ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133712025-04-30T06:13:34.771ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133722025-04-30T06:13:34.772ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133732025-04-30T06:13:34.778ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
133742025-04-30T06:13:34.787ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
133752025-04-30T06:13:34.787ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
133762025-04-30T06:13:34.787ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
133772025-04-30T06:13:34.788ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
133782025-04-30T06:13:34.789ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
133792025-04-30T06:13:34.794ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
133802025-04-30T06:13:34.794ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
133812025-04-30T06:13:34.795ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 50 e0 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 P............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56751 req_id = 522bad05-80bf-499b-8bd4-c60bc7c78cef
133822025-04-30T06:13:34.795ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20704,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56751 req_id = 522bad05-80bf-499b-8bd4-c60bc7c78cef
133832025-04-30T06:13:34.795ZDEBGomicron-dev (store): query key key = _mgs._tcp
133842025-04-30T06:13:34.795ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
133852025-04-30T06:13:34.796ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56751 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 522bad05-80bf-499b-8bd4-c60bc7c78cef
133862025-04-30T06:13:34.796ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2479, tv_nsec: 595514967 } })
133872025-04-30T06:13:34.796ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
133882025-04-30T06:13:34.797ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: fe 2a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .*..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49907 req_id = 4a9ad112-2635-48a1-8ceb-53e6eed8ec3a
133892025-04-30T06:13:34.797ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 65066,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49907 req_id = 4a9ad112-2635-48a1-8ceb-53e6eed8ec3a
133902025-04-30T06:13:34.797ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
133912025-04-30T06:13:34.798ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49907 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 4a9ad112-2635-48a1-8ceb-53e6eed8ec3a
133922025-04-30T06:13:34.845ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
133932025-04-30T06:13:34.845ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
133942025-04-30T06:13:34.845ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: f5 07 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50002 req_id = 2caca9f6-edcc-43c0-b819-371bb85e1723
133952025-04-30T06:13:34.845ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
133962025-04-30T06:13:34.845ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
133972025-04-30T06:13:34.845ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
133982025-04-30T06:13:34.846ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 62727,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50002 req_id = 2caca9f6-edcc-43c0-b819-371bb85e1723
133992025-04-30T06:13:34.846ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
134002025-04-30T06:13:34.846ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 62727, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50002 req_id = 2caca9f6-edcc-43c0-b819-371bb85e1723
134012025-04-30T06:13:34.846ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
134022025-04-30T06:13:34.847ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
134032025-04-30T06:13:34.847ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
134042025-04-30T06:13:34.847ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134052025-04-30T06:13:34.847ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
134062025-04-30T06:13:34.848ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 31 d8 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 1............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:64477 req_id = 6dfb238c-ab8e-45ae-a3a6-a07d2112d0b0
134072025-04-30T06:13:34.848ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 12760,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64477 req_id = 6dfb238c-ab8e-45ae-a3a6-a07d2112d0b0
134082025-04-30T06:13:34.848ZDEBGomicron-dev (store): query key key = _dendrite._tcp
134092025-04-30T06:13:34.849ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134102025-04-30T06:13:34.849ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64477 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 6dfb238c-ab8e-45ae-a3a6-a07d2112d0b0
134112025-04-30T06:13:34.851ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2479, tv_nsec: 649038720 } })
134122025-04-30T06:13:34.851ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
134132025-04-30T06:13:34.851ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9a7cb28-baab-454b-8826-c7e6554a6c08", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
134142025-04-30T06:13:34.851ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: b6 c7 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:49462 req_id = dfe16215-c3bb-4dda-9ae0-3d1995afe10b
134152025-04-30T06:13:34.851ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46791,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49462 req_id = dfe16215-c3bb-4dda-9ae0-3d1995afe10b
134162025-04-30T06:13:34.851ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134172025-04-30T06:13:34.851ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:49462 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = dfe16215-c3bb-4dda-9ae0-3d1995afe10b
134182025-04-30T06:13:34.851ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
134192025-04-30T06:13:34.852ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
134202025-04-30T06:13:34.853ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
134212025-04-30T06:13:34.854ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ff6fbaa-7cc6-4c2c-93b4-ef57cd003352", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134222025-04-30T06:13:34.854ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
134232025-04-30T06:13:34.857ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d781a73-31a3-4eaa-b938-3f329f2ce795", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
134242025-04-30T06:13:34.858ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
134252025-04-30T06:13:34.859ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
134262025-04-30T06:13:34.859ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
134272025-04-30T06:13:34.859ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
134282025-04-30T06:13:34.860ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134292025-04-30T06:13:34.861ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b80fcabb-8c6a-4710-ba42-514847c62b1c", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:33 GMT"} })
134302025-04-30T06:13:34.861ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
134312025-04-30T06:13:34.863ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
134322025-04-30T06:13:34.863ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
134332025-04-30T06:13:34.864ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "70ae56c2-5795-43ca-b86d-7667b027acbd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134342025-04-30T06:13:34.864ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
134352025-04-30T06:13:34.864ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
134362025-04-30T06:13:34.868ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb7e69ac-c281-4c58-a2e9-ae699aa586ff", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134372025-04-30T06:13:34.869ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
134382025-04-30T06:13:34.870ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dd0a859-a464-4cc8-a87d-ccbea57b50e9", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134392025-04-30T06:13:34.870ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
134402025-04-30T06:13:34.872ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
134412025-04-30T06:13:34.872ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dbc0504-7b05-42ce-9f86-3e2f2e0cb72f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134422025-04-30T06:13:34.872ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
134432025-04-30T06:13:34.873ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 33 82 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 3............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:54627 req_id = 0860569d-fec6-4148-9b96-e07af1520018
134442025-04-30T06:13:34.873ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13186,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54627 req_id = 0860569d-fec6-4148-9b96-e07af1520018
134452025-04-30T06:13:34.873ZDEBGomicron-dev (store): query key key = _dendrite._tcp
134462025-04-30T06:13:34.873ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134472025-04-30T06:13:34.873ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:54627 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 0860569d-fec6-4148-9b96-e07af1520018
134482025-04-30T06:13:34.874ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2479, tv_nsec: 673037135 } })
134492025-04-30T06:13:34.874ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
134502025-04-30T06:13:34.874ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 1a 2e 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58422 req_id = 2cc4b6e3-b7e7-481c-bd3c-be6f0dc52763
134512025-04-30T06:13:34.874ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 6702,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58422 req_id = 2cc4b6e3-b7e7-481c-bd3c-be6f0dc52763
134522025-04-30T06:13:34.875ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134532025-04-30T06:13:34.875ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58422 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 2cc4b6e3-b7e7-481c-bd3c-be6f0dc52763
134542025-04-30T06:13:34.875ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
134552025-04-30T06:13:34.876ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd2142ed-8f5f-4327-a6ba-3067a6831e88", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134562025-04-30T06:13:34.877ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
134572025-04-30T06:13:34.880ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b550c409-d841-4eb2-8bfe-aa55bf924cac", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134582025-04-30T06:13:34.880ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
134592025-04-30T06:13:34.881ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8318bd09-8945-4098-909d-7c152a0540fd", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134602025-04-30T06:13:34.881ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
134612025-04-30T06:13:34.882ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ad7e70b-0604-4f2c-869b-afb929e3b48d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134622025-04-30T06:13:34.884ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
134632025-04-30T06:13:34.887ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc5c394d-ad93-4f1f-985e-29fb3bcd9b0b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134642025-04-30T06:13:34.888ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
134652025-04-30T06:13:34.890ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
134662025-04-30T06:13:34.893ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "660577d3-5d10-42eb-a6e6-6be7e6d290ce", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134672025-04-30T06:13:34.894ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
134682025-04-30T06:13:34.895ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: ff 00 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:50655 req_id = 04b97cf2-732f-46ab-8e01-1ad73f875011
134692025-04-30T06:13:34.895ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 65280,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50655 req_id = 04b97cf2-732f-46ab-8e01-1ad73f875011
134702025-04-30T06:13:34.895ZDEBGomicron-dev (store): query key key = _dendrite._tcp
134712025-04-30T06:13:34.896ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134722025-04-30T06:13:34.896ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50655 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 04b97cf2-732f-46ab-8e01-1ad73f875011
134732025-04-30T06:13:34.897ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2479, tv_nsec: 696192992 } })
134742025-04-30T06:13:34.897ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
134752025-04-30T06:13:34.897ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 33 fe 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 3...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54283 req_id = 95e0f698-b3bb-4032-b5a9-e034b00dff1a
134762025-04-30T06:13:34.898ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13310,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54283 req_id = 95e0f698-b3bb-4032-b5a9-e034b00dff1a
134772025-04-30T06:13:34.898ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af00181e-0b5c-4e8e-b87a-eeeb44eb9f08", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134782025-04-30T06:13:34.898ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
134792025-04-30T06:13:34.898ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
134802025-04-30T06:13:34.898ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54283 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 95e0f698-b3bb-4032-b5a9-e034b00dff1a
134812025-04-30T06:13:34.898ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
134822025-04-30T06:13:34.900ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e13c1d31-e0ad-437d-9dd6-6777882fab6b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134832025-04-30T06:13:34.900ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
134842025-04-30T06:13:34.902ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9a2269f-9109-4b33-8f99-e2b1bf0c8f45", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134852025-04-30T06:13:34.902ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
134862025-04-30T06:13:34.906ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0449ae2c-dd06-41ef-b78e-087c9c66dfc3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134872025-04-30T06:13:34.906ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
134882025-04-30T06:13:34.908ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "08601e72-75c4-4e91-93e6-fbfb20c1b4fb", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134892025-04-30T06:13:34.909ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
134902025-04-30T06:13:34.910ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cddf538-be54-4746-a4a8-3466a979d196", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134912025-04-30T06:13:34.910ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
134922025-04-30T06:13:34.913ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce3642a0-037d-476a-9a13-29ee852f09b9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134932025-04-30T06:13:34.913ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
134942025-04-30T06:13:34.916ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "539bc2a7-898a-4147-8de7-574ca6fde258", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134952025-04-30T06:13:34.917ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
134962025-04-30T06:13:34.918ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6328b221-41ee-4c82-907a-b8f7732b8334", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134972025-04-30T06:13:34.918ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
134982025-04-30T06:13:34.920ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca780593-8f3b-4552-8ddb-fa2627bfa9e1", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
134992025-04-30T06:13:34.920ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
135002025-04-30T06:13:34.924ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f49a9230-3d84-4e21-aa0d-fc20e0c46d9f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135012025-04-30T06:13:34.924ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
135022025-04-30T06:13:34.924ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7df11ea1-8d00-4cfb-bf02-f6de616c71d6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135032025-04-30T06:13:34.924ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
135042025-04-30T06:13:34.927ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d682f6d-f671-4b33-8951-1d0b217c2d9f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135052025-04-30T06:13:34.927ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
135062025-04-30T06:13:34.928ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a310456d-1161-42f9-9fa8-2b7b85a488c2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135072025-04-30T06:13:34.928ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
135082025-04-30T06:13:34.930ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8aa7cc21-4788-4dc6-9794-710e226ac622", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135092025-04-30T06:13:34.930ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
135102025-04-30T06:13:34.932ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0cc7c8d0-e982-4e16-bb02-0fe51da895c4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135112025-04-30T06:13:34.932ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
135122025-04-30T06:13:34.933ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "635c2fce-9664-4370-ad00-bbbfcb821629", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135132025-04-30T06:13:34.933ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
135142025-04-30T06:13:34.934ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e68301a4-ac35-4f8b-9890-b1bdbf7883b9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135152025-04-30T06:13:34.934ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
135162025-04-30T06:13:34.935ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "892ef1c6-8795-49fe-9862-89a0e4b59e46", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135172025-04-30T06:13:34.936ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
135182025-04-30T06:13:34.936ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a02223ce-96fc-4ba7-8d93-de4aff70169f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135192025-04-30T06:13:34.936ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
135202025-04-30T06:13:34.940ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2c6c536-31dc-4235-9426-7ac062930124", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135212025-04-30T06:13:34.941ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
135222025-04-30T06:13:34.944ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4df3d485-0804-4b5f-8896-ab917a8ba869", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135232025-04-30T06:13:34.945ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
135242025-04-30T06:13:34.947ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "323405c9-aeef-4f29-a3d8-74b07bd14e48", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135252025-04-30T06:13:34.947ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
135262025-04-30T06:13:34.950ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e5ebbfb-50ba-4e58-a714-e44446c72486", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135272025-04-30T06:13:34.950ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
135282025-04-30T06:13:34.953ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ecfe8c1-2e56-49c2-9fe1-421536707c79", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135292025-04-30T06:13:34.953ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
135302025-04-30T06:13:34.960ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9970e4de-9c74-4a3e-8800-d550eb60bb94", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135312025-04-30T06:13:34.960ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
135322025-04-30T06:13:34.964ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b3b4284-0cba-4f89-aed7-16c5b95e08a8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135332025-04-30T06:13:34.964ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
135342025-04-30T06:13:34.965ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9961c95e-d5e9-41f5-babf-adcb76618308", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135352025-04-30T06:13:34.965ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
135362025-04-30T06:13:34.967ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a776483a-65ce-4f6e-a08b-a4ff9be4988e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135372025-04-30T06:13:34.967ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
135382025-04-30T06:13:34.968ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd2f701e-c4f8-445f-8f8a-f9c5a35e2cef", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135392025-04-30T06:13:34.968ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
135402025-04-30T06:13:34.973ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88c4f87e-7f80-461a-8319-7f232a9c3df6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
135412025-04-30T06:13:34.973ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
135422025-04-30T06:13:34.973ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
135432025-04-30T06:13:34.973ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
135442025-04-30T06:13:34.973ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135452025-04-30T06:13:34.973ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
135462025-04-30T06:13:34.974ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
135472025-04-30T06:13:34.976ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135482025-04-30T06:13:34.984ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
135492025-04-30T06:13:34.984ZDEBGomicron-dev (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
135502025-04-30T06:13:34.984ZDEBGomicron-dev (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
135512025-04-30T06:13:34.984ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
135522025-04-30T06:13:34.984ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
135532025-04-30T06:13:34.985ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
135542025-04-30T06:13:34.985ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
135552025-04-30T06:13:34.985ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
135562025-04-30T06:13:34.985ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135572025-04-30T06:13:34.986ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
135582025-04-30T06:13:34.986ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
135592025-04-30T06:13:34.989ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135602025-04-30T06:13:34.994ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
135612025-04-30T06:13:34.994ZDEBGomicron-dev (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
135622025-04-30T06:13:34.994ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
135632025-04-30T06:13:34.994ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
135642025-04-30T06:13:34.994ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135652025-04-30T06:13:34.994ZDEBGomicron-dev (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
135662025-04-30T06:13:34.994ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
135672025-04-30T06:13:34.996ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
135682025-04-30T06:13:34.998ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
135692025-04-30T06:13:35.025ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
135702025-04-30T06:13:35.026ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
135712025-04-30T06:13:35.026ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135722025-04-30T06:13:35.026ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
135732025-04-30T06:13:35.027ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
135742025-04-30T06:13:35.031ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
135752025-04-30T06:13:35.031ZDEBGomicron-dev (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
135762025-04-30T06:13:35.031ZDEBGomicron-dev (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
135772025-04-30T06:13:35.031ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
135782025-04-30T06:13:35.033ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
135792025-04-30T06:13:35.073ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
135802025-04-30T06:13:35.073ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
135812025-04-30T06:13:35.074ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:45030 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
135822025-04-30T06:13:35.075ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:45030 req_id = 24eb9e76-f142-4e17-9ace-ab64ce7693ca sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
135832025-04-30T06:13:35.075ZINFOomicron-dev (dropshot): request completed latency_us = 507 local_addr = [::1]:61881 method = GET remote_addr = [::1]:45030 req_id = 24eb9e76-f142-4e17-9ace-ab64ce7693ca response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
135842025-04-30T06:13:35.076ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "24eb9e76-f142-4e17-9ace-ab64ce7693ca", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
135852025-04-30T06:13:35.077ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135862025-04-30T06:13:35.079ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
135872025-04-30T06:13:35.079ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
135882025-04-30T06:13:35.079ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
135892025-04-30T06:13:35.082ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
135902025-04-30T06:13:35.082ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
135912025-04-30T06:13:35.082ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
135922025-04-30T06:13:35.082ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
135932025-04-30T06:13:35.083ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
135942025-04-30T06:13:35.086ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
135952025-04-30T06:13:35.089ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
135962025-04-30T06:13:35.089ZDEBGomicron-dev (results-sink): inserting 6 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
135972025-04-30T06:13:35.089ZDEBGomicron-dev (clickhouse-client): unrolling 6 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
135982025-04-30T06:13:35.090ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
135992025-04-30T06:13:35.090ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
136002025-04-30T06:13:35.091ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136012025-04-30T06:13:35.091ZDEBGomicron-dev (results-sink-cluster): inserting 6 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
136022025-04-30T06:13:35.095ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
136032025-04-30T06:13:35.096ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136042025-04-30T06:13:35.096ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136052025-04-30T06:13:35.096ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 12 sql = INSERT INTO oximeter.fields_string FORMAT Native
136062025-04-30T06:13:35.097ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136072025-04-30T06:13:35.101ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 12 table_name = fields_string
136082025-04-30T06:13:35.101ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 5 sql = INSERT INTO oximeter.fields_u16 FORMAT Native
136092025-04-30T06:13:35.101ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136102025-04-30T06:13:35.101ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
136112025-04-30T06:13:35.102ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136122025-04-30T06:13:35.104ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 5 table_name = fields_u16
136132025-04-30T06:13:35.104ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 5 sql = INSERT INTO oximeter.fields_uuid FORMAT Native
136142025-04-30T06:13:35.109ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 5 table_name = fields_uuid
136152025-04-30T06:13:35.109ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 19 n_rows = 5 sql = INSERT INTO oximeter.measurements_histogramu64 FORMAT Native
136162025-04-30T06:13:35.113ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 5 table_name = measurements_histogramu64
136172025-04-30T06:13:35.113ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
136182025-04-30T06:13:35.120ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
136192025-04-30T06:13:35.120ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
136202025-04-30T06:13:35.155ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
136212025-04-30T06:13:35.158ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
136222025-04-30T06:13:35.158ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 386.696156ms iteration = 49 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
136232025-04-30T06:13:35.158ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 50 reason = Signaled
136242025-04-30T06:13:35.158ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
136252025-04-30T06:13:35.158ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136262025-04-30T06:13:35.158ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136272025-04-30T06:13:35.158ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136282025-04-30T06:13:35.161ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136292025-04-30T06:13:35.164ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136302025-04-30T06:13:35.169ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
136312025-04-30T06:13:35.169ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
136322025-04-30T06:13:35.169ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
136332025-04-30T06:13:35.170ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
136342025-04-30T06:13:35.192ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
136352025-04-30T06:13:35.208ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
136362025-04-30T06:13:35.208ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
136372025-04-30T06:13:35.209ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 16 d2 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62162 req_id = 8b19f8a1-6d07-4afb-aa8f-42ac897d557a
136382025-04-30T06:13:35.209ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5842,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62162 req_id = 8b19f8a1-6d07-4afb-aa8f-42ac897d557a
136392025-04-30T06:13:35.210ZDEBGomicron-dev (store): query key key = _mgs._tcp
136402025-04-30T06:13:35.210ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
136412025-04-30T06:13:35.210ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62162 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8b19f8a1-6d07-4afb-aa8f-42ac897d557a
136422025-04-30T06:13:35.211ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2480, tv_nsec: 9631157 } })
136432025-04-30T06:13:35.211ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
136442025-04-30T06:13:35.212ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 20 4c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e L..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54580 req_id = 633b0f30-5d4e-4615-af09-dd290eb01094
136452025-04-30T06:13:35.212ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8268,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54580 req_id = 633b0f30-5d4e-4615-af09-dd290eb01094
136462025-04-30T06:13:35.212ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
136472025-04-30T06:13:35.212ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54580 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 633b0f30-5d4e-4615-af09-dd290eb01094
136482025-04-30T06:13:35.222ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
136492025-04-30T06:13:35.258ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
136502025-04-30T06:13:35.258ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: b6 b4 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:53214 req_id = 1fbc66fd-51b4-4396-a915-d01e2696960a
136512025-04-30T06:13:35.258ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 46772,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53214 req_id = 1fbc66fd-51b4-4396-a915-d01e2696960a
136522025-04-30T06:13:35.259ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
136532025-04-30T06:13:35.259ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 46772, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:53214 req_id = 1fbc66fd-51b4-4396-a915-d01e2696960a
136542025-04-30T06:13:35.259ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
136552025-04-30T06:13:35.259ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
136562025-04-30T06:13:35.259ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
136572025-04-30T06:13:35.261ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0a871546-a787-4742-b5f7-857c9f6c7b0f", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136582025-04-30T06:13:35.262ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
136592025-04-30T06:13:35.266ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "126f11ba-b7e1-4e18-ba9c-130c7e151d8c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136602025-04-30T06:13:35.266ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
136612025-04-30T06:13:35.270ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "376bd303-246c-4573-afe8-1d6749171b38", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136622025-04-30T06:13:35.270ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
136632025-04-30T06:13:35.277ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
136642025-04-30T06:13:35.277ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
136652025-04-30T06:13:35.277ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136662025-04-30T06:13:35.277ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
136672025-04-30T06:13:35.277ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136682025-04-30T06:13:35.278ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba07af5c-24cf-4bcb-8fec-9cf1973d67fb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136692025-04-30T06:13:35.278ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
136702025-04-30T06:13:35.278ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a8086cc-590a-4f26-adeb-1c3143c2e446", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
136712025-04-30T06:13:35.278ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
136722025-04-30T06:13:35.278ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f449fbb-4d62-4820-9eb2-4ef1e1609281", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
136732025-04-30T06:13:35.278ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
136742025-04-30T06:13:35.280ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03ac4c1c-4dff-4090-bc43-e8987f102b34", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
136752025-04-30T06:13:35.281ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
136762025-04-30T06:13:35.282ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
136772025-04-30T06:13:35.283ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b0cc5b2-b143-41a0-bcf7-b1baf7eff4d3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
136782025-04-30T06:13:35.283ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
136792025-04-30T06:13:35.288ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "806522f0-34ff-402c-bea8-ccb255821c50", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136802025-04-30T06:13:35.288ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
136812025-04-30T06:13:35.291ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
136822025-04-30T06:13:35.292ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136832025-04-30T06:13:35.292ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
136842025-04-30T06:13:35.292ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136852025-04-30T06:13:35.296ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dcef2892-a209-4539-ba7c-805aa5662b86", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136862025-04-30T06:13:35.297ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
136872025-04-30T06:13:35.297ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
136882025-04-30T06:13:35.297ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
136892025-04-30T06:13:35.306ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
136902025-04-30T06:13:35.309ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "977744e5-f1b0-4cd4-b678-6efd11b7267a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136912025-04-30T06:13:35.309ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
136922025-04-30T06:13:35.311ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "31417105-0187-41f1-976a-3c560b2cdb50", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136932025-04-30T06:13:35.311ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
136942025-04-30T06:13:35.312ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fe136519-bbcc-4ff7-bab8-84c6f583ab6b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136952025-04-30T06:13:35.312ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
136962025-04-30T06:13:35.315ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a6971523-f58a-45f2-be00-e0f4a3240188", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136972025-04-30T06:13:35.315ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
136982025-04-30T06:13:35.324ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "31117d90-117b-49fe-8b99-04ca217302dc", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
136992025-04-30T06:13:35.324ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
137002025-04-30T06:13:35.325ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f16b6cf7-deeb-4ef4-a998-1952ae87b69b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137012025-04-30T06:13:35.325ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
137022025-04-30T06:13:35.325ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "95a48557-e305-45e1-a83a-91c19756f1e7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137032025-04-30T06:13:35.325ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
137042025-04-30T06:13:35.328ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "37a793f2-a1d4-4900-8399-73112a367ba0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137052025-04-30T06:13:35.329ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
137062025-04-30T06:13:35.331ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9444ed80-5279-4b27-8a1c-fa618fabd642", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137072025-04-30T06:13:35.336ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
137082025-04-30T06:13:35.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e4d5f4d-915b-43fc-83b1-099ec69fcbc3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137092025-04-30T06:13:35.336ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
137102025-04-30T06:13:35.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5b5b425-0f76-42fe-a123-822ab7907c52", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137112025-04-30T06:13:35.336ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
137122025-04-30T06:13:35.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a2632654-ed2f-4a05-a2c5-6ce2427569c0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137132025-04-30T06:13:35.336ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
137142025-04-30T06:13:35.336ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29baeda6-b430-4b2a-a7ab-9c920e906ca1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137152025-04-30T06:13:35.337ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
137162025-04-30T06:13:35.338ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9b33870-abc7-453b-be1d-ba715cb4748f", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137172025-04-30T06:13:35.338ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
137182025-04-30T06:13:35.341ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd35f704-1530-4cc6-b958-077a17ff540e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137192025-04-30T06:13:35.341ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
137202025-04-30T06:13:35.343ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "93851e44-24ba-4439-8fe2-1b180f619fc3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137212025-04-30T06:13:35.343ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
137222025-04-30T06:13:35.345ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
137232025-04-30T06:13:35.387ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "18fcc8e0-069f-4964-9f34-7848bc84869f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137242025-04-30T06:13:35.388ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
137252025-04-30T06:13:35.391ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "84206913-7c2b-4ad2-aaed-4a271e6a962b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137262025-04-30T06:13:35.391ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
137272025-04-30T06:13:35.393ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "55057368-791e-4855-90e5-4c47597a6f2e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137282025-04-30T06:13:35.394ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
137292025-04-30T06:13:35.397ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
137302025-04-30T06:13:35.398ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
137312025-04-30T06:13:35.398ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
137322025-04-30T06:13:35.398ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
137332025-04-30T06:13:35.402ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
137342025-04-30T06:13:35.402ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137352025-04-30T06:13:35.403ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a5085936-8b3a-4510-9901-4c0b0367149f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137362025-04-30T06:13:35.404ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
137372025-04-30T06:13:35.406ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b6f42e4-a303-4c72-89d2-8d785d822094", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
137382025-04-30T06:13:35.406ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
137392025-04-30T06:13:35.409ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
137402025-04-30T06:13:35.409ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
137412025-04-30T06:13:35.409ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
137422025-04-30T06:13:35.409ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "292e0c42-7f5e-4c32-bb7a-6831ccbae486", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137432025-04-30T06:13:35.409ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
137442025-04-30T06:13:35.414ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
137452025-04-30T06:13:35.414ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b75672e-b3aa-4915-adca-5c03fd81fc53", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
137462025-04-30T06:13:35.414ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
137472025-04-30T06:13:35.414ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e543073b-7675-4d7a-bc9d-9b4b52e06a48", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137482025-04-30T06:13:35.414ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
137492025-04-30T06:13:35.414ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
137502025-04-30T06:13:35.414ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
137512025-04-30T06:13:35.415ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
137522025-04-30T06:13:35.416ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6437bc36-676f-474e-a623-98454ca3fed3", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137532025-04-30T06:13:35.416ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
137542025-04-30T06:13:35.418ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eac9fc10-c111-4545-a4c2-c763b9379a1e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137552025-04-30T06:13:35.419ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
137562025-04-30T06:13:35.420ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4920393-06fc-492a-94c8-95d456521397", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137572025-04-30T06:13:35.421ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
137582025-04-30T06:13:35.423ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b8a850df-1a7d-4b78-b804-6e7d34921df6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137592025-04-30T06:13:35.423ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
137602025-04-30T06:13:35.425ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6394b144-2d0b-4809-81e3-c7d5b9f41e01", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137612025-04-30T06:13:35.425ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
137622025-04-30T06:13:35.427ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b903905d-3430-4b3f-ac71-6fcecf5699e7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137632025-04-30T06:13:35.427ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
137642025-04-30T06:13:35.429ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db415deb-8eb4-43a3-946c-ec304ba8e0dc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137652025-04-30T06:13:35.429ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
137662025-04-30T06:13:35.430ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "db97855a-b323-4161-a455-47713287e762", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137672025-04-30T06:13:35.430ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
137682025-04-30T06:13:35.432ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "34f57b78-6832-4703-b1b2-1d12cc38e0af", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137692025-04-30T06:13:35.432ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
137702025-04-30T06:13:35.433ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37c8858d-1ca8-432d-9c44-73562ced363a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137712025-04-30T06:13:35.433ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
137722025-04-30T06:13:35.434ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e2c7889-65fe-43a7-9515-a33e42fac219", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
137732025-04-30T06:13:35.434ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
137742025-04-30T06:13:35.434ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
137752025-04-30T06:13:35.434ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
137762025-04-30T06:13:35.434ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
137772025-04-30T06:13:35.435ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
137782025-04-30T06:13:35.441ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137792025-04-30T06:13:35.446ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
137802025-04-30T06:13:35.452ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
137812025-04-30T06:13:35.452ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
137822025-04-30T06:13:35.453ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
137832025-04-30T06:13:35.453ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
137842025-04-30T06:13:35.453ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
137852025-04-30T06:13:35.453ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
137862025-04-30T06:13:35.457ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
137872025-04-30T06:13:35.465ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
137882025-04-30T06:13:35.466ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
137892025-04-30T06:13:35.466ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
137902025-04-30T06:13:35.467ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
137912025-04-30T06:13:35.497ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
137922025-04-30T06:13:35.497ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
137932025-04-30T06:13:35.497ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
137942025-04-30T06:13:35.497ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
137952025-04-30T06:13:35.498ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
137962025-04-30T06:13:35.559ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
137972025-04-30T06:13:35.559ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
137982025-04-30T06:13:35.559ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:54093 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
137992025-04-30T06:13:35.559ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:54093 req_id = 11932701-b034-444d-88c0-3a469b4d4b0f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
138002025-04-30T06:13:35.559ZINFOomicron-dev (dropshot): request completed latency_us = 803 local_addr = [::1]:61881 method = GET remote_addr = [::1]:54093 req_id = 11932701-b034-444d-88c0-3a469b4d4b0f response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
138012025-04-30T06:13:35.560ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "11932701-b034-444d-88c0-3a469b4d4b0f", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
138022025-04-30T06:13:35.562ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138032025-04-30T06:13:35.562ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
138042025-04-30T06:13:35.562ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
138052025-04-30T06:13:35.562ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
138062025-04-30T06:13:35.565ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
138072025-04-30T06:13:35.565ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138082025-04-30T06:13:35.565ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138092025-04-30T06:13:35.565ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138102025-04-30T06:13:35.566ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138112025-04-30T06:13:35.568ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
138122025-04-30T06:13:35.568ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138132025-04-30T06:13:35.568ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
138142025-04-30T06:13:35.568ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138152025-04-30T06:13:35.569ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138162025-04-30T06:13:35.572ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
138172025-04-30T06:13:35.572ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138182025-04-30T06:13:35.572ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138192025-04-30T06:13:35.573ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138202025-04-30T06:13:35.576ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138212025-04-30T06:13:35.576ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
138222025-04-30T06:13:35.576ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138232025-04-30T06:13:35.600ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
138242025-04-30T06:13:35.600ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
138252025-04-30T06:13:35.601ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 442.596956ms iteration = 50 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
138262025-04-30T06:13:35.602ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 51 reason = Signaled
138272025-04-30T06:13:35.602ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
138282025-04-30T06:13:35.602ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138292025-04-30T06:13:35.602ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138302025-04-30T06:13:35.602ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138312025-04-30T06:13:35.603ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138322025-04-30T06:13:35.607ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138332025-04-30T06:13:35.608ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
138342025-04-30T06:13:35.611ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
138352025-04-30T06:13:35.611ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
138362025-04-30T06:13:35.612ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
138372025-04-30T06:13:35.612ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
138382025-04-30T06:13:35.618ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
138392025-04-30T06:13:35.618ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
138402025-04-30T06:13:35.618ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 6e 7a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 nz..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64863 req_id = 0b73da8e-acb8-48ce-bfbe-72f13527f636
138412025-04-30T06:13:35.619ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28282,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64863 req_id = 0b73da8e-acb8-48ce-bfbe-72f13527f636
138422025-04-30T06:13:35.619ZDEBGomicron-dev (store): query key key = _mgs._tcp
138432025-04-30T06:13:35.619ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
138442025-04-30T06:13:35.619ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64863 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 0b73da8e-acb8-48ce-bfbe-72f13527f636
138452025-04-30T06:13:35.620ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2480, tv_nsec: 418999116 } })
138462025-04-30T06:13:35.622ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
138472025-04-30T06:13:35.623ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 3f f4 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ?...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56840 req_id = 66b9d0f7-77bc-42e1-a2f8-f7f6672fa2ec
138482025-04-30T06:13:35.623ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16372,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56840 req_id = 66b9d0f7-77bc-42e1-a2f8-f7f6672fa2ec
138492025-04-30T06:13:35.624ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
138502025-04-30T06:13:35.624ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56840 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 66b9d0f7-77bc-42e1-a2f8-f7f6672fa2ec
138512025-04-30T06:13:35.664ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
138522025-04-30T06:13:35.664ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
138532025-04-30T06:13:35.664ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138542025-04-30T06:13:35.664ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
138552025-04-30T06:13:35.667ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138562025-04-30T06:13:35.667ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
138572025-04-30T06:13:35.669ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: cf 3f 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .?..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:50539 req_id = d08c3d4f-b45e-4a52-9fac-2fce7d51461f
138582025-04-30T06:13:35.669ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53055,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50539 req_id = d08c3d4f-b45e-4a52-9fac-2fce7d51461f
138592025-04-30T06:13:35.670ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
138602025-04-30T06:13:35.670ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 53055, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:50539 req_id = d08c3d4f-b45e-4a52-9fac-2fce7d51461f
138612025-04-30T06:13:35.670ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
138622025-04-30T06:13:35.670ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
138632025-04-30T06:13:35.670ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
138642025-04-30T06:13:35.671ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
138652025-04-30T06:13:35.674ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c2f5d5f-d819-4541-be4b-78baf83a5750", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
138662025-04-30T06:13:35.675ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
138672025-04-30T06:13:35.676ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
138682025-04-30T06:13:35.676ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138692025-04-30T06:13:35.676ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
138702025-04-30T06:13:35.676ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e237b054-5922-4c7e-8c15-1ab2808169d4", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
138712025-04-30T06:13:35.677ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138722025-04-30T06:13:35.677ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
138732025-04-30T06:13:35.679ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
138742025-04-30T06:13:35.679ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
138752025-04-30T06:13:35.680ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
138762025-04-30T06:13:35.680ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "35ceb56f-001c-402b-a5d1-0eb5638c57ce", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
138772025-04-30T06:13:35.680ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
138782025-04-30T06:13:35.683ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c12d0587-d5f9-4637-863a-0568a24347c9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
138792025-04-30T06:13:35.683ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
138802025-04-30T06:13:35.686ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc3bebd4-008c-42ef-8eea-aaafb121de94", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
138812025-04-30T06:13:35.686ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
138822025-04-30T06:13:35.689ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "62860496-20b9-480f-a73c-f0458afc4bd2", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
138832025-04-30T06:13:35.689ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
138842025-04-30T06:13:35.693ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "82ec96c2-1eff-42ea-866e-de7b48aefe34", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
138852025-04-30T06:13:35.693ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
138862025-04-30T06:13:35.698ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98cb1d07-de4b-486e-8983-5461951a71e3", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
138872025-04-30T06:13:35.698ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
138882025-04-30T06:13:35.699ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "75a90ccb-4977-4509-a3b1-39e650a95188", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
138892025-04-30T06:13:35.700ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
138902025-04-30T06:13:35.702ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f62b1b3-a94c-401e-80ad-ef1b726d3851", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
138912025-04-30T06:13:35.702ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
138922025-04-30T06:13:35.705ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "45a9d94c-1c25-46a2-953b-737088e258fa", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
138932025-04-30T06:13:35.705ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
138942025-04-30T06:13:35.706ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6e59e44-f486-49d4-b0a8-7bf73cc6be6b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
138952025-04-30T06:13:35.707ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
138962025-04-30T06:13:35.708ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cffcff2e-02b0-4bcf-bff1-69b4cbbac45f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
138972025-04-30T06:13:35.708ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
138982025-04-30T06:13:35.709ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
138992025-04-30T06:13:35.711ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d66f17b1-bee9-46a3-ae79-012112e7610d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
139002025-04-30T06:13:35.711ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
139012025-04-30T06:13:35.719ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "98cf2f2a-b2ac-4134-b36a-5cddbc29efd7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139022025-04-30T06:13:35.720ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
139032025-04-30T06:13:35.720ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5aeeed1a-f66a-4748-9dca-6f8bb95d77ab", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139042025-04-30T06:13:35.720ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
139052025-04-30T06:13:35.720ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
139062025-04-30T06:13:35.720ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
139072025-04-30T06:13:35.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
139082025-04-30T06:13:35.720ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
139092025-04-30T06:13:35.720ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
139102025-04-30T06:13:35.720ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
139112025-04-30T06:13:35.722ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b281b14-7063-475c-aa32-7e7c369895bb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139122025-04-30T06:13:35.722ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
139132025-04-30T06:13:35.724ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9301197a-fbe4-4062-b71e-5748b0fa20b2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139142025-04-30T06:13:35.724ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
139152025-04-30T06:13:35.726ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9b02fd1-1098-477c-9900-725df70fff10", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139162025-04-30T06:13:35.726ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
139172025-04-30T06:13:35.727ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e44763a7-3bb9-42d0-b001-1fe6f4f15e09", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139182025-04-30T06:13:35.727ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
139192025-04-30T06:13:35.728ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3583cc78-ca47-41a7-bdb0-047abfa0ded5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139202025-04-30T06:13:35.728ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
139212025-04-30T06:13:35.729ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c75809b-e3d5-4ab9-b689-55792efb0f31", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139222025-04-30T06:13:35.729ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
139232025-04-30T06:13:35.730ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "518e5488-5291-4e1f-9c63-4bcf5856724f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139242025-04-30T06:13:35.730ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
139252025-04-30T06:13:35.732ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "779f3209-6585-4c00-959b-88910881e5c6", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139262025-04-30T06:13:35.732ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
139272025-04-30T06:13:35.734ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "20c5f6e6-9aa3-45bc-9d55-29c645406076", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139282025-04-30T06:13:35.734ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
139292025-04-30T06:13:35.738ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1c1b561-3448-452c-b112-79c9f2d128d5", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139302025-04-30T06:13:35.740ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
139312025-04-30T06:13:35.745ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "167c440d-4fa2-43e8-83e9-6c208f9146f7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139322025-04-30T06:13:35.745ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
139332025-04-30T06:13:35.749ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ba7e1f47-6624-422d-99e2-083f384c1b90", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139342025-04-30T06:13:35.749ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
139352025-04-30T06:13:35.754ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e6430dbf-cf2f-423b-86dd-9eece7041439", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139362025-04-30T06:13:35.755ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
139372025-04-30T06:13:35.757ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d7a5c6f-c41f-4793-b7b4-e26457dcca90", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139382025-04-30T06:13:35.758ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
139392025-04-30T06:13:35.758ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0609b3b-d650-435e-a9ba-fdd40fa7893c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139402025-04-30T06:13:35.759ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
139412025-04-30T06:13:35.760ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cac6fff-7da9-4c8d-b138-640a8ff19f22", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139422025-04-30T06:13:35.760ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
139432025-04-30T06:13:35.761ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
139442025-04-30T06:13:35.762ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
139452025-04-30T06:13:35.762ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139462025-04-30T06:13:35.762ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
139472025-04-30T06:13:35.763ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a41271d-f09d-459c-b6fe-d8f8250903d7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
139482025-04-30T06:13:35.763ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
139492025-04-30T06:13:35.764ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139502025-04-30T06:13:35.764ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "37c8dfb3-7188-4600-987c-6c80d1436a12", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139512025-04-30T06:13:35.765ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
139522025-04-30T06:13:35.766ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "34c1e217-2106-4768-ac1d-bb2ff3ab9677", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139532025-04-30T06:13:35.766ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
139542025-04-30T06:13:35.768ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139552025-04-30T06:13:35.769ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bae5f1b-b8c8-44b6-a83c-92e06eba7212", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139562025-04-30T06:13:35.769ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
139572025-04-30T06:13:35.772ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "70bd6edb-fbbe-418c-bca1-96f74606cfb6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139582025-04-30T06:13:35.772ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
139592025-04-30T06:13:35.774ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb2e951e-9663-41ef-96cd-6810c9205d56", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139602025-04-30T06:13:35.774ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
139612025-04-30T06:13:35.776ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "60407d43-cf52-4150-92e8-5b6b3ece4dcd", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139622025-04-30T06:13:35.776ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
139632025-04-30T06:13:35.777ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e042d60-6976-4f0d-9518-f97e9199810d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139642025-04-30T06:13:35.778ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
139652025-04-30T06:13:35.778ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139662025-04-30T06:13:35.778ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
139672025-04-30T06:13:35.778ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
139682025-04-30T06:13:35.778ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139692025-04-30T06:13:35.780ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bbd958b0-59a5-472d-8db3-0605a1f06197", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139702025-04-30T06:13:35.781ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
139712025-04-30T06:13:35.782ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "405447f1-7ce4-4bb6-ac69-e67ff2342378", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139722025-04-30T06:13:35.784ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
139732025-04-30T06:13:35.788ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
139742025-04-30T06:13:35.788ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
139752025-04-30T06:13:35.789ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "49d8fe7d-1148-4e3a-8c3e-40106be572e4", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139762025-04-30T06:13:35.789ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
139772025-04-30T06:13:35.789ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
139782025-04-30T06:13:35.791ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c3ac5fa-21b4-4687-88f1-9f68a5ea9500", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:34 GMT"} })
139792025-04-30T06:13:35.792ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
139802025-04-30T06:13:35.794ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "acdc5606-71ab-4811-bbe1-b7a66fa80e3d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
139812025-04-30T06:13:35.794ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
139822025-04-30T06:13:35.794ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
139832025-04-30T06:13:35.795ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
139842025-04-30T06:13:35.795ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139852025-04-30T06:13:35.796ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139862025-04-30T06:13:35.802ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139872025-04-30T06:13:35.809ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
139882025-04-30T06:13:35.809ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
139892025-04-30T06:13:35.809ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
139902025-04-30T06:13:35.809ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
139912025-04-30T06:13:35.809ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139922025-04-30T06:13:35.811ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139932025-04-30T06:13:35.818ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
139942025-04-30T06:13:35.823ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
139952025-04-30T06:13:35.823ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
139962025-04-30T06:13:35.823ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
139972025-04-30T06:13:35.824ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
139982025-04-30T06:13:35.843ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
139992025-04-30T06:13:35.880ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
140002025-04-30T06:13:35.880ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
140012025-04-30T06:13:35.880ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:50923 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
140022025-04-30T06:13:35.881ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:50923 req_id = 17f96a58-74d3-44c7-9463-9934460b90f7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
140032025-04-30T06:13:35.881ZINFOomicron-dev (dropshot): request completed latency_us = 641 local_addr = [::1]:61881 method = GET remote_addr = [::1]:50923 req_id = 17f96a58-74d3-44c7-9463-9934460b90f7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
140042025-04-30T06:13:35.882ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "17f96a58-74d3-44c7-9463-9934460b90f7", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140052025-04-30T06:13:35.884ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
140062025-04-30T06:13:35.884ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
140072025-04-30T06:13:35.884ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
140082025-04-30T06:13:35.888ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
140092025-04-30T06:13:35.888ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
140102025-04-30T06:13:35.889ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
140112025-04-30T06:13:35.889ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140122025-04-30T06:13:35.890ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
140132025-04-30T06:13:35.893ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140142025-04-30T06:13:35.895ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
140152025-04-30T06:13:35.895ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
140162025-04-30T06:13:35.895ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140172025-04-30T06:13:35.895ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
140182025-04-30T06:13:35.896ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140192025-04-30T06:13:35.898ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
140202025-04-30T06:13:35.899ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
140212025-04-30T06:13:35.900ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140222025-04-30T06:13:35.900ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
140232025-04-30T06:13:35.904ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140242025-04-30T06:13:35.911ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
140252025-04-30T06:13:35.911ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140262025-04-30T06:13:35.911ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
140272025-04-30T06:13:35.912ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140282025-04-30T06:13:35.928ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
140292025-04-30T06:13:35.929ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
140302025-04-30T06:13:35.929ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 327.227568ms iteration = 51 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
140312025-04-30T06:13:35.929ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 52 reason = Signaled
140322025-04-30T06:13:35.929ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
140332025-04-30T06:13:35.929ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
140342025-04-30T06:13:35.929ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
140352025-04-30T06:13:35.929ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140362025-04-30T06:13:35.930ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
140372025-04-30T06:13:35.933ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
140382025-04-30T06:13:35.936ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
140392025-04-30T06:13:35.936ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
140402025-04-30T06:13:35.937ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
140412025-04-30T06:13:35.940ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
140422025-04-30T06:13:35.940ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
140432025-04-30T06:13:35.940ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
140442025-04-30T06:13:35.941ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
140452025-04-30T06:13:35.952ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
140462025-04-30T06:13:35.952ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
140472025-04-30T06:13:35.955ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 10 53 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .S..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:62337 req_id = 41d6c336-1b7f-47e1-912a-8eed8d1e2317
140482025-04-30T06:13:35.955ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 4179,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62337 req_id = 41d6c336-1b7f-47e1-912a-8eed8d1e2317
140492025-04-30T06:13:35.955ZDEBGomicron-dev (store): query key key = _mgs._tcp
140502025-04-30T06:13:35.955ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
140512025-04-30T06:13:35.955ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62337 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 41d6c336-1b7f-47e1-912a-8eed8d1e2317
140522025-04-30T06:13:35.964ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2480, tv_nsec: 757933650 } })
140532025-04-30T06:13:35.964ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
140542025-04-30T06:13:35.964ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6c 39 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e l9..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56515 req_id = 440d8a2a-98a9-45f3-af42-f2b5c547f658
140552025-04-30T06:13:35.964ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27705,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56515 req_id = 440d8a2a-98a9-45f3-af42-f2b5c547f658
140562025-04-30T06:13:35.964ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
140572025-04-30T06:13:35.964ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56515 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 440d8a2a-98a9-45f3-af42-f2b5c547f658
140582025-04-30T06:13:36.025ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
140592025-04-30T06:13:36.034ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
140602025-04-30T06:13:36.034ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 4f 95 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c O............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64674 req_id = 839f5a09-65d8-466b-b414-8fb06f4b126c
140612025-04-30T06:13:36.035ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 20373,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64674 req_id = 839f5a09-65d8-466b-b414-8fb06f4b126c
140622025-04-30T06:13:36.035ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
140632025-04-30T06:13:36.035ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 20373, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64674 req_id = 839f5a09-65d8-466b-b414-8fb06f4b126c
140642025-04-30T06:13:36.035ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
140652025-04-30T06:13:36.035ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
140662025-04-30T06:13:36.035ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
140672025-04-30T06:13:36.037ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f49a555d-9f1b-43ea-976e-bf57091de5aa", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140682025-04-30T06:13:36.038ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
140692025-04-30T06:13:36.039ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "651c93e7-91cf-4ab5-be94-e6aad222b20b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
140702025-04-30T06:13:36.039ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
140712025-04-30T06:13:36.042ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9d0ac3b3-50ef-4e76-a07b-cbe9e08577a3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140722025-04-30T06:13:36.042ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
140732025-04-30T06:13:36.043ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a656d53-0d37-4ccd-8c75-050342408291", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140742025-04-30T06:13:36.043ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
140752025-04-30T06:13:36.045ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa9e10fd-ba13-4792-a90d-ac302c16b50f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140762025-04-30T06:13:36.045ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
140772025-04-30T06:13:36.046ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fde7b387-066b-4450-8b46-e7a13fca342e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140782025-04-30T06:13:36.046ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
140792025-04-30T06:13:36.048ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "af021ff1-5100-427c-83f1-67e3daf0d5c1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140802025-04-30T06:13:36.048ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
140812025-04-30T06:13:36.050ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "28c6aa06-b9dd-4f66-80b0-72633db85c98", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140822025-04-30T06:13:36.050ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
140832025-04-30T06:13:36.051ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "043dcff1-b841-46ec-85cb-255ccd071c8c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140842025-04-30T06:13:36.051ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
140852025-04-30T06:13:36.052ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e04dc2e3-503a-4ab3-9fae-67919ef66354", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140862025-04-30T06:13:36.052ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
140872025-04-30T06:13:36.054ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c80f8296-5199-49a4-a8d4-a57eb3506c66", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
140882025-04-30T06:13:36.054ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
140892025-04-30T06:13:36.055ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "18d612ed-ad96-434c-a990-faeee1ece8dc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140902025-04-30T06:13:36.055ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
140912025-04-30T06:13:36.057ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f2758a3-5c0b-45ab-8b6f-d4e46ed87234", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140922025-04-30T06:13:36.057ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
140932025-04-30T06:13:36.058ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "764ab0c4-bd7b-43b8-9f28-4f05bf273317", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140942025-04-30T06:13:36.059ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
140952025-04-30T06:13:36.061ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b13ad7b-cff5-4eff-ad4a-f32a5083f2ae", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140962025-04-30T06:13:36.061ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
140972025-04-30T06:13:36.062ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c41d4dc-49e8-4a77-b809-9abcaee9e4bf", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
140982025-04-30T06:13:36.062ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
140992025-04-30T06:13:36.064ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7295e264-58a6-45fe-914e-7731eebc6127", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141002025-04-30T06:13:36.064ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
141012025-04-30T06:13:36.066ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ed6329fb-2825-4f1d-b6c8-0b243a6a0782", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141022025-04-30T06:13:36.067ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
141032025-04-30T06:13:36.067ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80a7170b-0dce-4644-90bd-5948a02009d0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141042025-04-30T06:13:36.068ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
141052025-04-30T06:13:36.068ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f41440c1-76ea-4d27-bd47-f7a4e7810e0f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141062025-04-30T06:13:36.069ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
141072025-04-30T06:13:36.071ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "421583ff-cdc8-4550-a016-981f31bbff02", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141082025-04-30T06:13:36.071ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
141092025-04-30T06:13:36.072ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b6a999a-0104-4bc3-87da-54f296308129", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141102025-04-30T06:13:36.072ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
141112025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86e35600-08ff-4b5e-9942-ead32c1992dc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141122025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
141132025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "209cd621-aee6-4e1b-ab73-ddcce48324b7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141142025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
141152025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1df39846-7e42-4ba3-a1c9-107948d928f4", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141162025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
141172025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2f26414-22b6-4772-8f76-c48204a29ffa", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141182025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
141192025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9f92831-b623-4d3c-9d3d-dde2d8b8a340", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141202025-04-30T06:13:36.081ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
141212025-04-30T06:13:36.085ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb2b1156-b182-49b9-a464-38c815a835a6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141222025-04-30T06:13:36.085ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
141232025-04-30T06:13:36.085ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "76b24dd0-5b7b-47a5-8f6e-60a691ff4b34", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141242025-04-30T06:13:36.085ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
141252025-04-30T06:13:36.086ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c04101f9-547e-4e54-a5c5-c118eb70e69a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141262025-04-30T06:13:36.086ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
141272025-04-30T06:13:36.087ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "71d1d915-d64e-4c80-b903-65b767fd795c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141282025-04-30T06:13:36.087ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
141292025-04-30T06:13:36.090ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "581f9cad-42bc-496e-9a15-73fa0d92484f", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141302025-04-30T06:13:36.090ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
141312025-04-30T06:13:36.090ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
141322025-04-30T06:13:36.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
141332025-04-30T06:13:36.090ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "165e2810-55cc-496c-8ea8-743f76131e06", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141342025-04-30T06:13:36.090ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
141352025-04-30T06:13:36.090ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
141362025-04-30T06:13:36.091ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
141372025-04-30T06:13:36.095ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
141382025-04-30T06:13:36.095ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141392025-04-30T06:13:36.096ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141402025-04-30T06:13:36.096ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
141412025-04-30T06:13:36.096ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8aa6e3b-408d-4f91-8d6a-765287da07e5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141422025-04-30T06:13:36.096ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
141432025-04-30T06:13:36.097ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
141442025-04-30T06:13:36.097ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
141452025-04-30T06:13:36.097ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141462025-04-30T06:13:36.098ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f11b6b8-aac5-4d4f-879e-fc5c12a1bee1", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141472025-04-30T06:13:36.098ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
141482025-04-30T06:13:36.102ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141492025-04-30T06:13:36.103ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "10355cd0-ad96-49ae-9f4a-9d149133cd80", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141502025-04-30T06:13:36.103ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
141512025-04-30T06:13:36.103ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
141522025-04-30T06:13:36.107ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
141532025-04-30T06:13:36.109ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1248f0c-b7cb-4968-a3ed-354f156ae2ea", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141542025-04-30T06:13:36.110ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
141552025-04-30T06:13:36.111ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a86ca5b1-e47d-45fc-b026-a21b8167dfd2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141562025-04-30T06:13:36.111ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
141572025-04-30T06:13:36.112ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
141582025-04-30T06:13:36.112ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141592025-04-30T06:13:36.112ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
141602025-04-30T06:13:36.113ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141612025-04-30T06:13:36.114ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6452519a-f2c5-445a-a984-71da0d6f5895", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141622025-04-30T06:13:36.114ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
141632025-04-30T06:13:36.116ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
141642025-04-30T06:13:36.116ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
141652025-04-30T06:13:36.117ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
141662025-04-30T06:13:36.120ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a993497-9091-4b05-a6ed-c4ce808a7454", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141672025-04-30T06:13:36.120ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
141682025-04-30T06:13:36.122ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb012d54-7588-411f-96bb-e9c1053a1bbf", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141692025-04-30T06:13:36.123ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
141702025-04-30T06:13:36.124ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "54e42292-22ea-4b49-a222-b3896d1b3b1c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141712025-04-30T06:13:36.125ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
141722025-04-30T06:13:36.127ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7fec5b05-8250-42d1-ae87-eb22d5a34267", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141732025-04-30T06:13:36.127ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
141742025-04-30T06:13:36.129ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e6073fd-9157-4dd8-9e24-56aacd80cc41", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
141752025-04-30T06:13:36.129ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
141762025-04-30T06:13:36.130ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2feec22d-9bd9-4d9d-851a-6f2f811f2229", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
141772025-04-30T06:13:36.130ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
141782025-04-30T06:13:36.130ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141792025-04-30T06:13:36.130ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141802025-04-30T06:13:36.130ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141812025-04-30T06:13:36.131ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141822025-04-30T06:13:36.134ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141832025-04-30T06:13:36.138ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
141842025-04-30T06:13:36.138ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
141852025-04-30T06:13:36.138ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
141862025-04-30T06:13:36.138ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141872025-04-30T06:13:36.138ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141882025-04-30T06:13:36.139ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141892025-04-30T06:13:36.142ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
141902025-04-30T06:13:36.151ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
141912025-04-30T06:13:36.151ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
141922025-04-30T06:13:36.151ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
141932025-04-30T06:13:36.152ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
141942025-04-30T06:13:36.159ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
141952025-04-30T06:13:36.191ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
141962025-04-30T06:13:36.196ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
141972025-04-30T06:13:36.197ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
141982025-04-30T06:13:36.197ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:34660 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
141992025-04-30T06:13:36.199ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:34660 req_id = 6e93e540-aab9-4cb7-aa35-2896a1c27416 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
142002025-04-30T06:13:36.199ZINFOomicron-dev (dropshot): request completed latency_us = 562 local_addr = [::1]:61881 method = GET remote_addr = [::1]:34660 req_id = 6e93e540-aab9-4cb7-aa35-2896a1c27416 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
142012025-04-30T06:13:36.199ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e93e540-aab9-4cb7-aa35-2896a1c27416", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142022025-04-30T06:13:36.202ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
142032025-04-30T06:13:36.202ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
142042025-04-30T06:13:36.202ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
142052025-04-30T06:13:36.209ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
142062025-04-30T06:13:36.210ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
142072025-04-30T06:13:36.210ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
142082025-04-30T06:13:36.210ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
142092025-04-30T06:13:36.210ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
142102025-04-30T06:13:36.210ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
142112025-04-30T06:13:36.210ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142122025-04-30T06:13:36.210ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
142132025-04-30T06:13:36.211ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
142142025-04-30T06:13:36.212ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142152025-04-30T06:13:36.214ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142162025-04-30T06:13:36.215ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142172025-04-30T06:13:36.218ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
142182025-04-30T06:13:36.218ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
142192025-04-30T06:13:36.218ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
142202025-04-30T06:13:36.219ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
142212025-04-30T06:13:36.221ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
142222025-04-30T06:13:36.221ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142232025-04-30T06:13:36.221ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
142242025-04-30T06:13:36.222ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142252025-04-30T06:13:36.223ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142262025-04-30T06:13:36.223ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
142272025-04-30T06:13:36.224ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142282025-04-30T06:13:36.245ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
142292025-04-30T06:13:36.246ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
142302025-04-30T06:13:36.246ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 316.986936ms iteration = 52 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
142312025-04-30T06:13:36.246ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 53 reason = Signaled
142322025-04-30T06:13:36.246ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
142332025-04-30T06:13:36.246ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
142342025-04-30T06:13:36.246ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
142352025-04-30T06:13:36.246ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
142362025-04-30T06:13:36.247ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
142372025-04-30T06:13:36.256ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142382025-04-30T06:13:36.260ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
142392025-04-30T06:13:36.260ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
142402025-04-30T06:13:36.260ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
142412025-04-30T06:13:36.260ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
142422025-04-30T06:13:36.265ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
142432025-04-30T06:13:36.265ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
142442025-04-30T06:13:36.266ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 1c 79 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .y..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64848 req_id = 5db8bb76-5c3c-4b68-a73d-0f7f07593c70
142452025-04-30T06:13:36.266ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7289,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64848 req_id = 5db8bb76-5c3c-4b68-a73d-0f7f07593c70
142462025-04-30T06:13:36.267ZDEBGomicron-dev (store): query key key = _mgs._tcp
142472025-04-30T06:13:36.267ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
142482025-04-30T06:13:36.267ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
142492025-04-30T06:13:36.267ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64848 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5db8bb76-5c3c-4b68-a73d-0f7f07593c70
142502025-04-30T06:13:36.268ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2481, tv_nsec: 67096407 } })
142512025-04-30T06:13:36.268ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
142522025-04-30T06:13:36.271ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 9a 42 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .B..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:61038 req_id = d59d1b65-4db3-4b98-ae63-e5264a8e9172
142532025-04-30T06:13:36.271ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39490,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61038 req_id = d59d1b65-4db3-4b98-ae63-e5264a8e9172
142542025-04-30T06:13:36.271ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
142552025-04-30T06:13:36.271ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:61038 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = d59d1b65-4db3-4b98-ae63-e5264a8e9172
142562025-04-30T06:13:36.310ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
142572025-04-30T06:13:36.311ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 45 b3 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c E............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61671 req_id = 7eed1b6d-1975-409c-9295-b94c35362305
142582025-04-30T06:13:36.311ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17843,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61671 req_id = 7eed1b6d-1975-409c-9295-b94c35362305
142592025-04-30T06:13:36.311ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
142602025-04-30T06:13:36.312ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 17843, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61671 req_id = 7eed1b6d-1975-409c-9295-b94c35362305
142612025-04-30T06:13:36.312ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
142622025-04-30T06:13:36.313ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
142632025-04-30T06:13:36.313ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
142642025-04-30T06:13:36.317ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "24fd4470-759a-48be-a943-17e3c7b31adb", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142652025-04-30T06:13:36.318ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
142662025-04-30T06:13:36.318ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
142672025-04-30T06:13:36.318ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
142682025-04-30T06:13:36.318ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142692025-04-30T06:13:36.318ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
142702025-04-30T06:13:36.319ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142712025-04-30T06:13:36.323ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "860d71e7-7d41-457e-83df-b5ec58faf73c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142722025-04-30T06:13:36.324ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
142732025-04-30T06:13:36.324ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
142742025-04-30T06:13:36.328ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb7b5a50-5d3a-4758-a831-f66b943c672b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142752025-04-30T06:13:36.328ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
142762025-04-30T06:13:36.330ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d0b83469-640e-4e8a-a7a5-f410587f8951", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142772025-04-30T06:13:36.330ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
142782025-04-30T06:13:36.332ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cc5e64d-6429-4255-9c28-5a9eb341b294", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142792025-04-30T06:13:36.332ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
142802025-04-30T06:13:36.334ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
142812025-04-30T06:13:36.334ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142822025-04-30T06:13:36.334ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
142832025-04-30T06:13:36.334ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142842025-04-30T06:13:36.335ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1c9fa44-1082-44d9-85d0-9cd2c1cec188", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142852025-04-30T06:13:36.335ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
142862025-04-30T06:13:36.338ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
142872025-04-30T06:13:36.338ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
142882025-04-30T06:13:36.338ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9223d8ca-5deb-4b9b-ad6b-7d10c69720df", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142892025-04-30T06:13:36.339ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
142902025-04-30T06:13:36.339ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
142912025-04-30T06:13:36.342ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "76ddeb25-e00f-4a72-ac0f-6d53af04ee21", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142922025-04-30T06:13:36.342ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
142932025-04-30T06:13:36.343ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "14c3cdb0-08c0-4eb3-9590-da3fa9ec87f4", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142942025-04-30T06:13:36.343ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
142952025-04-30T06:13:36.344ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbc05ce8-01e6-44fe-9c3f-d3f31b0806d1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142962025-04-30T06:13:36.344ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
142972025-04-30T06:13:36.346ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bdc44fdb-a1c0-45b1-ae34-297551c7eabe", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
142982025-04-30T06:13:36.346ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
142992025-04-30T06:13:36.347ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb9e0313-a177-407f-8904-3a4d741a4f37", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143002025-04-30T06:13:36.347ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
143012025-04-30T06:13:36.349ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb585fff-fb5f-45f8-a763-6eb24bdb0637", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143022025-04-30T06:13:36.350ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
143032025-04-30T06:13:36.352ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1723dc3a-8b12-42be-9193-f423e55fc197", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143042025-04-30T06:13:36.352ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
143052025-04-30T06:13:36.355ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e0c08ad-01b2-4336-a9f4-3d5599a3c8b7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143062025-04-30T06:13:36.355ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
143072025-04-30T06:13:36.358ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "43d9da2c-51b9-47eb-9a1d-3e697f5412f9", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143082025-04-30T06:13:36.358ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
143092025-04-30T06:13:36.361ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa193b13-3fe9-46a6-acca-3c2b22c8631c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143102025-04-30T06:13:36.362ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
143112025-04-30T06:13:36.364ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21c4047e-235a-4b29-a25b-edd494b65f80", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143122025-04-30T06:13:36.365ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
143132025-04-30T06:13:36.367ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef2c3b91-018b-4bda-957f-f67bf10291ab", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143142025-04-30T06:13:36.367ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
143152025-04-30T06:13:36.368ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
143162025-04-30T06:13:36.369ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4aeb5a93-26e0-4fd6-a819-16e8f62de959", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143172025-04-30T06:13:36.369ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
143182025-04-30T06:13:36.371ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3dd3fc63-4486-4858-af85-c07ba271b6ca", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143192025-04-30T06:13:36.371ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
143202025-04-30T06:13:36.373ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "dc9bd068-d393-4e17-bafb-92924b5b3dc7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
143212025-04-30T06:13:36.373ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
143222025-04-30T06:13:36.374ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "593a896a-25fd-41fc-8019-e30c67490075", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143232025-04-30T06:13:36.375ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
143242025-04-30T06:13:36.376ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8e99ba8e-a396-4e50-af9a-2fcc9a1018db", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143252025-04-30T06:13:36.377ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
143262025-04-30T06:13:36.379ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f216800-069d-4a78-891c-9c8eece3e436", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143272025-04-30T06:13:36.379ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
143282025-04-30T06:13:36.381ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "845ca255-cfc1-4a0b-9f49-a7c91755c5ef", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143292025-04-30T06:13:36.381ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
143302025-04-30T06:13:36.383ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e86f729-9bfb-42d8-8252-5652079dd19c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143312025-04-30T06:13:36.384ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
143322025-04-30T06:13:36.390ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee295e5e-71e4-407c-8fab-687a7f09d4f1", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143332025-04-30T06:13:36.390ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
143342025-04-30T06:13:36.392ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8474a1d4-5ed6-4fc8-bb5e-da745481a075", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143352025-04-30T06:13:36.392ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
143362025-04-30T06:13:36.394ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dbbd29bc-bdee-46d5-a8fd-cafce5679d50", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143372025-04-30T06:13:36.394ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
143382025-04-30T06:13:36.395ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "68100c6b-e777-4c27-925f-b5fbf18a7842", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143392025-04-30T06:13:36.395ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
143402025-04-30T06:13:36.397ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ea36d05-8807-43b1-a782-42841dc0dc30", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143412025-04-30T06:13:36.397ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
143422025-04-30T06:13:36.399ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf5ed533-5080-4465-bad8-79202626f05b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143432025-04-30T06:13:36.399ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
143442025-04-30T06:13:36.400ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "333720d8-c741-4572-8310-647f7923a08b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143452025-04-30T06:13:36.401ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
143462025-04-30T06:13:36.402ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c120a67-f3d3-4215-8c7a-db3d7d1d3907", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143472025-04-30T06:13:36.403ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
143482025-04-30T06:13:36.406ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc9ebcd9-56bb-4637-bcaa-c5cc272f836a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143492025-04-30T06:13:36.406ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
143502025-04-30T06:13:36.408ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "63145ef2-882d-43a0-b2ee-105fb8502317", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143512025-04-30T06:13:36.408ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
143522025-04-30T06:13:36.410ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ada5ca61-7ea1-444e-94ab-e7a3e064756d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143532025-04-30T06:13:36.411ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
143542025-04-30T06:13:36.413ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ddb265b-9194-40d5-9333-89e0781ff20b", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143552025-04-30T06:13:36.413ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
143562025-04-30T06:13:36.416ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d044b18-f9b4-4600-a818-748c9f1fee27", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143572025-04-30T06:13:36.416ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
143582025-04-30T06:13:36.418ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c1b25d0-b725-4064-b6fd-44f318057d0e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143592025-04-30T06:13:36.418ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
143602025-04-30T06:13:36.419ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "30e98575-0c5d-4c37-b6ea-2413fb5fa67c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143612025-04-30T06:13:36.419ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
143622025-04-30T06:13:36.420ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
143632025-04-30T06:13:36.420ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143642025-04-30T06:13:36.420ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
143652025-04-30T06:13:36.420ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
143662025-04-30T06:13:36.420ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aba1687e-5b2a-496d-a027-4a3478fc7065", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143672025-04-30T06:13:36.421ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
143682025-04-30T06:13:36.422ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
143692025-04-30T06:13:36.423ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "683940d1-461b-46fb-9794-8e15780f17bb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
143702025-04-30T06:13:36.424ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143712025-04-30T06:13:36.424ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
143722025-04-30T06:13:36.426ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8a7d1a5-c7c4-4b55-b7f0-eec248cb7420", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
143732025-04-30T06:13:36.426ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
143742025-04-30T06:13:36.426ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143752025-04-30T06:13:36.426ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143762025-04-30T06:13:36.427ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143772025-04-30T06:13:36.430ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143782025-04-30T06:13:36.433ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
143792025-04-30T06:13:36.433ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
143802025-04-30T06:13:36.433ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
143812025-04-30T06:13:36.434ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
143822025-04-30T06:13:36.437ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143832025-04-30T06:13:36.441ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
143842025-04-30T06:13:36.441ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
143852025-04-30T06:13:36.442ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
143862025-04-30T06:13:36.442ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143872025-04-30T06:13:36.442ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143882025-04-30T06:13:36.442ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143892025-04-30T06:13:36.446ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
143902025-04-30T06:13:36.446ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
143912025-04-30T06:13:36.447ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
143922025-04-30T06:13:36.447ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
143932025-04-30T06:13:36.459ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
143942025-04-30T06:13:36.459ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
143952025-04-30T06:13:36.459ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
143962025-04-30T06:13:36.460ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
143972025-04-30T06:13:36.514ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
143982025-04-30T06:13:36.566ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
143992025-04-30T06:13:36.566ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
144002025-04-30T06:13:36.566ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
144012025-04-30T06:13:36.566ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
144022025-04-30T06:13:36.567ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
144032025-04-30T06:13:36.596ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
144042025-04-30T06:13:36.596ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
144052025-04-30T06:13:36.597ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:59477 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
144062025-04-30T06:13:36.599ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:59477 req_id = 15860c49-69b3-42c3-879c-40b3d62f8029 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
144072025-04-30T06:13:36.600ZINFOomicron-dev (dropshot): request completed latency_us = 704 local_addr = [::1]:61881 method = GET remote_addr = [::1]:59477 req_id = 15860c49-69b3-42c3-879c-40b3d62f8029 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
144082025-04-30T06:13:36.601ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "15860c49-69b3-42c3-879c-40b3d62f8029", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
144092025-04-30T06:13:36.602ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144102025-04-30T06:13:36.610ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
144112025-04-30T06:13:36.610ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
144122025-04-30T06:13:36.610ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
144132025-04-30T06:13:36.610ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
144142025-04-30T06:13:36.614ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
144152025-04-30T06:13:36.614ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
144162025-04-30T06:13:36.614ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
144172025-04-30T06:13:36.614ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
144182025-04-30T06:13:36.614ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
144192025-04-30T06:13:36.614ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
144202025-04-30T06:13:36.619ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
144212025-04-30T06:13:36.620ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
144222025-04-30T06:13:36.620ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
144232025-04-30T06:13:36.620ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144242025-04-30T06:13:36.623ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
144252025-04-30T06:13:36.627ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144262025-04-30T06:13:36.632ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
144272025-04-30T06:13:36.632ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
144282025-04-30T06:13:36.632ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144292025-04-30T06:13:36.633ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
144302025-04-30T06:13:36.658ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
144312025-04-30T06:13:36.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
144322025-04-30T06:13:36.680ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
144332025-04-30T06:13:36.686ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 738c9155-39a4-4b8a-8aae-1858ad11e85f uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
144342025-04-30T06:13:36.687ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 738c9155-39a4-4b8a-8aae-1858ad11e85f resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
144352025-04-30T06:13:36.687ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 738c9155-39a4-4b8a-8aae-1858ad11e85f roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
144362025-04-30T06:13:36.692ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
144372025-04-30T06:13:36.692ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
144382025-04-30T06:13:36.692ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 445.018881ms iteration = 53 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
144392025-04-30T06:13:36.692ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 54 reason = Signaled
144402025-04-30T06:13:36.692ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
144412025-04-30T06:13:36.692ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
144422025-04-30T06:13:36.692ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
144432025-04-30T06:13:36.692ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144442025-04-30T06:13:36.692ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
144452025-04-30T06:13:36.701ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144462025-04-30T06:13:36.702ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 738c9155-39a4-4b8a-8aae-1858ad11e85f resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
144472025-04-30T06:13:36.706ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
144482025-04-30T06:13:36.706ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
144492025-04-30T06:13:36.706ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
144502025-04-30T06:13:36.706ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
144512025-04-30T06:13:36.710ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
144522025-04-30T06:13:36.710ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
144532025-04-30T06:13:36.710ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
144542025-04-30T06:13:36.710ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
144552025-04-30T06:13:36.710ZINFOomicron-dev (dropshot_internal): request completed latency_us = 23309 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 738c9155-39a4-4b8a-8aae-1858ad11e85f response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
144562025-04-30T06:13:36.710ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "738c9155-39a4-4b8a-8aae-1858ad11e85f", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
144572025-04-30T06:13:36.710ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
144582025-04-30T06:13:36.710ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
144592025-04-30T06:13:36.710ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a58c17b-bb97-4fa8-8370-f1002ca51f9a uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
144602025-04-30T06:13:36.711ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a58c17b-bb97-4fa8-8370-f1002ca51f9a resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
144612025-04-30T06:13:36.711ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a58c17b-bb97-4fa8-8370-f1002ca51f9a roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
144622025-04-30T06:13:36.712ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
144632025-04-30T06:13:36.712ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
144642025-04-30T06:13:36.712ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a58c17b-bb97-4fa8-8370-f1002ca51f9a resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
144652025-04-30T06:13:36.713ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
144662025-04-30T06:13:36.716ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 4b 73 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 Ks..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61627 req_id = d1ca2cef-de0a-4a7e-9456-05c709b44ce1
144672025-04-30T06:13:36.716ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19315,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61627 req_id = d1ca2cef-de0a-4a7e-9456-05c709b44ce1
144682025-04-30T06:13:36.716ZDEBGomicron-dev (store): query key key = _mgs._tcp
144692025-04-30T06:13:36.716ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144702025-04-30T06:13:36.716ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61627 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = d1ca2cef-de0a-4a7e-9456-05c709b44ce1
144712025-04-30T06:13:36.717ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2481, tv_nsec: 516137191 } })
144722025-04-30T06:13:36.717ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
144732025-04-30T06:13:36.717ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: db 8e 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53504 req_id = fa460e48-1fa7-4491-88d0-96f9b2ec64c1
144742025-04-30T06:13:36.718ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56206,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53504 req_id = fa460e48-1fa7-4491-88d0-96f9b2ec64c1
144752025-04-30T06:13:36.718ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
144762025-04-30T06:13:36.718ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53504 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = fa460e48-1fa7-4491-88d0-96f9b2ec64c1
144772025-04-30T06:13:36.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
144782025-04-30T06:13:36.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
144792025-04-30T06:13:36.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
144802025-04-30T06:13:36.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
144812025-04-30T06:13:36.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
144822025-04-30T06:13:36.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
144832025-04-30T06:13:36.721ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
144842025-04-30T06:13:36.721ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
144852025-04-30T06:13:36.724ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
144862025-04-30T06:13:36.725ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
144872025-04-30T06:13:36.757ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
144882025-04-30T06:13:36.758ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: d2 a0 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:60650 req_id = 9f55c70c-237b-4270-a49f-82cd01e1b3e5
144892025-04-30T06:13:36.758ZINFOomicron-dev (dropshot_internal): request completed latency_us = 47679 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a58c17b-bb97-4fa8-8370-f1002ca51f9a response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
144902025-04-30T06:13:36.758ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 53920,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60650 req_id = 9f55c70c-237b-4270-a49f-82cd01e1b3e5
144912025-04-30T06:13:36.758ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
144922025-04-30T06:13:36.758ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 53920, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:60650 req_id = 9f55c70c-237b-4270-a49f-82cd01e1b3e5
144932025-04-30T06:13:36.759ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a58c17b-bb97-4fa8-8370-f1002ca51f9a", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
144942025-04-30T06:13:36.759ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
144952025-04-30T06:13:36.759ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
144962025-04-30T06:13:36.759ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
144972025-04-30T06:13:36.759ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
144982025-04-30T06:13:36.759ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
144992025-04-30T06:13:36.759ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
145002025-04-30T06:13:36.761ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
145012025-04-30T06:13:36.761ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
145022025-04-30T06:13:36.761ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac14f26c-d9ad-4c3a-ae7b-91bc76c43b49", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145032025-04-30T06:13:36.761ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
145042025-04-30T06:13:36.763ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
145052025-04-30T06:13:36.763ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c5040ce-8f01-4b95-bd6c-a3231e3ccc4c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145062025-04-30T06:13:36.764ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
145072025-04-30T06:13:36.766ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "380fe585-03ff-4dd4-a4e4-7907ddc527e6", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145082025-04-30T06:13:36.766ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
145092025-04-30T06:13:36.768ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "facee460-879f-4e5a-acba-98f648f8cf69", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145102025-04-30T06:13:36.768ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
145112025-04-30T06:13:36.770ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51800e3e-653f-40f7-a0c5-43965fce8eef", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145122025-04-30T06:13:36.770ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
145132025-04-30T06:13:36.773ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c825663-61e2-413e-b464-49f295bb5775", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145142025-04-30T06:13:36.773ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
145152025-04-30T06:13:36.778ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffe1bb4d-7ddf-49eb-a75b-b4c964b4d387", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145162025-04-30T06:13:36.778ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
145172025-04-30T06:13:36.784ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "82052a36-b3aa-4c0e-bf26-4d6ea2b4ee46", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145182025-04-30T06:13:36.785ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
145192025-04-30T06:13:36.787ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd1345ff-d1a1-4721-9b8b-36418f690065", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145202025-04-30T06:13:36.787ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
145212025-04-30T06:13:36.789ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "20804fba-1048-4d84-bdc1-16fb0bea928a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145222025-04-30T06:13:36.789ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
145232025-04-30T06:13:36.790ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "11103e32-fe06-42a7-a754-5bfb1a6f378d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145242025-04-30T06:13:36.791ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
145252025-04-30T06:13:36.793ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "415acc47-f4b5-40e4-a22e-a5d098c89ddf", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145262025-04-30T06:13:36.793ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
145272025-04-30T06:13:36.795ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "259076c7-fe6e-43e4-a8f1-56568477c69f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145282025-04-30T06:13:36.795ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
145292025-04-30T06:13:36.798ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c52c7cfa-d2d0-49df-96af-1ae2a61ae865", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145302025-04-30T06:13:36.798ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
145312025-04-30T06:13:36.814ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "17dd0a1b-a071-4362-a9f4-7a6a76313c69", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145322025-04-30T06:13:36.814ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
145332025-04-30T06:13:36.817ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc95939d-d16d-4829-9edb-ceb220690db5", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145342025-04-30T06:13:36.817ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
145352025-04-30T06:13:36.853ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "132fc823-b82c-4641-9aec-276425b69888", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145362025-04-30T06:13:36.853ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
145372025-04-30T06:13:36.860ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "895be6a6-35b7-4c72-a55a-e39e9637e85b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145382025-04-30T06:13:36.860ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
145392025-04-30T06:13:36.862ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
145402025-04-30T06:13:36.864ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b94c7e24-dbba-43d5-83d9-b13f0f2f23b6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145412025-04-30T06:13:36.864ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
145422025-04-30T06:13:36.866ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cad42c8-9ae7-4471-a2bd-7f4a4bdb42d6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145432025-04-30T06:13:36.866ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
145442025-04-30T06:13:36.869ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6652cd5-910d-4449-bf12-d57fd1e7bcbf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145452025-04-30T06:13:36.870ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
145462025-04-30T06:13:36.872ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3589efc3-f949-4f35-a3e3-1ee29952d955", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145472025-04-30T06:13:36.872ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
145482025-04-30T06:13:36.873ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b1bf065-3f4f-43c8-b414-c6580aa8cd88", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145492025-04-30T06:13:36.873ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
145502025-04-30T06:13:36.879ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "58552602-89c1-4706-bb5c-62a5b66cba95", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145512025-04-30T06:13:36.879ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
145522025-04-30T06:13:36.879ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd73acf1-e6c8-4dee-a2c7-30053fc3b276", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145532025-04-30T06:13:36.879ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
145542025-04-30T06:13:36.884ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e11ccc82-6611-4dcd-a436-9581019e5ff0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:35 GMT"} })
145552025-04-30T06:13:36.884ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
145562025-04-30T06:13:36.886ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0bdd4cc-7a79-405b-b0f3-11de3e485a80", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145572025-04-30T06:13:36.886ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
145582025-04-30T06:13:36.889ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "edeb1e5f-c2ff-49dd-b018-2caad60969db", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145592025-04-30T06:13:36.889ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
145602025-04-30T06:13:36.891ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c1a39048-b9f4-4f99-bc8b-c42122a930da", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145612025-04-30T06:13:36.891ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
145622025-04-30T06:13:36.893ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "37b9c30b-aeef-46b2-8a80-579965d88daa", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145632025-04-30T06:13:36.893ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
145642025-04-30T06:13:36.894ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5e0d4bf4-3db9-469c-bb8b-74fbc7ed5019", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145652025-04-30T06:13:36.894ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
145662025-04-30T06:13:36.895ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "daf2ee56-4c69-4bed-8389-85e4db365adf", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145672025-04-30T06:13:36.895ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
145682025-04-30T06:13:36.896ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e111a659-8d53-410b-a9d0-958fd6d41808", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145692025-04-30T06:13:36.896ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
145702025-04-30T06:13:36.897ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b6218b2-fcff-478f-b9db-429b8b61f41d", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145712025-04-30T06:13:36.897ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
145722025-04-30T06:13:36.898ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "25d1e6e6-cd81-450e-9124-7b99c0fddd4b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145732025-04-30T06:13:36.899ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
145742025-04-30T06:13:36.905ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "319fb36b-3ec0-4346-a3f5-911ac7fe578c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145752025-04-30T06:13:36.907ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
145762025-04-30T06:13:36.911ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ce5e89da-b791-4ec1-a460-6b5762272135", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145772025-04-30T06:13:36.911ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
145782025-04-30T06:13:36.914ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9872efd9-b470-4a3f-965b-e530c7b1b5a9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145792025-04-30T06:13:36.915ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
145802025-04-30T06:13:36.916ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
145812025-04-30T06:13:36.916ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
145822025-04-30T06:13:36.916ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
145832025-04-30T06:13:36.916ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
145842025-04-30T06:13:36.918ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5d1cd7f-ab4e-41fc-be17-907ffb521976", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145852025-04-30T06:13:36.918ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
145862025-04-30T06:13:36.918ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
145872025-04-30T06:13:36.921ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8309f0b0-bba1-48fa-a369-c4fdb06360bc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145882025-04-30T06:13:36.921ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
145892025-04-30T06:13:36.923ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "275f7c83-3ad3-4209-8f69-b77ffd271d96", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145902025-04-30T06:13:36.923ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
145912025-04-30T06:13:36.924ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a72a420-f40e-4cb6-b1c5-b15b94687d5e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145922025-04-30T06:13:36.925ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
145932025-04-30T06:13:36.925ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3eda05b3-b4fd-4c1a-aa8d-7d0733abd089", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145942025-04-30T06:13:36.926ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
145952025-04-30T06:13:36.927ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d01e744e-914e-4f91-883b-5c5f7b2473f2", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145962025-04-30T06:13:36.927ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
145972025-04-30T06:13:36.927ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
145982025-04-30T06:13:36.930ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a098f60-a4fa-46ba-b2f8-17f3ad674157", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
145992025-04-30T06:13:36.931ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
146002025-04-30T06:13:36.931ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
146012025-04-30T06:13:36.931ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146022025-04-30T06:13:36.931ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146032025-04-30T06:13:36.932ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146042025-04-30T06:13:36.936ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
146052025-04-30T06:13:36.937ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
146062025-04-30T06:13:36.937ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
146072025-04-30T06:13:36.941ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
146082025-04-30T06:13:36.950ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146092025-04-30T06:13:36.955ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
146102025-04-30T06:13:36.955ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
146112025-04-30T06:13:36.955ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
146122025-04-30T06:13:36.955ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146132025-04-30T06:13:36.955ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146142025-04-30T06:13:36.956ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146152025-04-30T06:13:36.960ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146162025-04-30T06:13:36.962ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
146172025-04-30T06:13:36.962ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
146182025-04-30T06:13:36.963ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
146192025-04-30T06:13:36.964ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
146202025-04-30T06:13:36.964ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
146212025-04-30T06:13:36.964ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
146222025-04-30T06:13:36.964ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
146232025-04-30T06:13:36.988ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
146242025-04-30T06:13:36.988ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 3d 83 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 =............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:50689 req_id = cc17a3bb-1a3e-4120-bee8-23d8d2db59ea
146252025-04-30T06:13:36.988ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15747,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50689 req_id = cc17a3bb-1a3e-4120-bee8-23d8d2db59ea
146262025-04-30T06:13:36.988ZDEBGomicron-dev (store): query key key = _dendrite._tcp
146272025-04-30T06:13:36.989ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
146282025-04-30T06:13:36.989ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50689 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = cc17a3bb-1a3e-4120-bee8-23d8d2db59ea
146292025-04-30T06:13:36.989ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2481, tv_nsec: 788463443 } })
146302025-04-30T06:13:36.989ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
146312025-04-30T06:13:36.990ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 00 1f 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50743 req_id = 84219059-c059-4754-9b98-497be9a4d2ef
146322025-04-30T06:13:36.990ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50743 req_id = 84219059-c059-4754-9b98-497be9a4d2ef
146332025-04-30T06:13:36.990ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
146342025-04-30T06:13:36.990ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50743 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 84219059-c059-4754-9b98-497be9a4d2ef
146352025-04-30T06:13:36.990ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
146362025-04-30T06:13:37.019ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
146372025-04-30T06:13:37.020ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 75 f2 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 u............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:58343 req_id = 048d3c35-ba66-4ee1-af0c-f7472d59daf1
146382025-04-30T06:13:37.020ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 30194,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58343 req_id = 048d3c35-ba66-4ee1-af0c-f7472d59daf1
146392025-04-30T06:13:37.021ZDEBGomicron-dev (store): query key key = _dendrite._tcp
146402025-04-30T06:13:37.021ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
146412025-04-30T06:13:37.023ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58343 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 048d3c35-ba66-4ee1-af0c-f7472d59daf1
146422025-04-30T06:13:37.024ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2481, tv_nsec: 822066605 } })
146432025-04-30T06:13:37.024ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
146442025-04-30T06:13:37.024ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 34 8d 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e 4...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52067 req_id = 441ac4fa-34d4-45e1-81f8-866ba20efa35
146452025-04-30T06:13:37.029ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 13453,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52067 req_id = 441ac4fa-34d4-45e1-81f8-866ba20efa35
146462025-04-30T06:13:37.029ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
146472025-04-30T06:13:37.029ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52067 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 441ac4fa-34d4-45e1-81f8-866ba20efa35
146482025-04-30T06:13:37.029ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
146492025-04-30T06:13:37.036ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
146502025-04-30T06:13:37.041ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: ac dc 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:52265 req_id = a8c9fd54-4ba0-47b6-9d4d-ae2706d328fc
146512025-04-30T06:13:37.041ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44252,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52265 req_id = a8c9fd54-4ba0-47b6-9d4d-ae2706d328fc
146522025-04-30T06:13:37.041ZDEBGomicron-dev (store): query key key = _dendrite._tcp
146532025-04-30T06:13:37.041ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
146542025-04-30T06:13:37.042ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52265 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = a8c9fd54-4ba0-47b6-9d4d-ae2706d328fc
146552025-04-30T06:13:37.042ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2481, tv_nsec: 841862427 } })
146562025-04-30T06:13:37.043ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
146572025-04-30T06:13:37.053ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 6a e3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e j...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:56834 req_id = e0d0112e-274e-4fe6-a85e-8d0ef0a00942
146582025-04-30T06:13:37.053ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 27363,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56834 req_id = e0d0112e-274e-4fe6-a85e-8d0ef0a00942
146592025-04-30T06:13:37.053ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
146602025-04-30T06:13:37.054ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:56834 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e0d0112e-274e-4fe6-a85e-8d0ef0a00942
146612025-04-30T06:13:37.054ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
146622025-04-30T06:13:37.093ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
146632025-04-30T06:13:37.264ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
146642025-04-30T06:13:37.264ZDEBGomicron-dev (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
146652025-04-30T06:13:37.264ZDEBGomicron-dev (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
146662025-04-30T06:13:37.264ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
146672025-04-30T06:13:37.351ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
146682025-04-30T06:13:37.351ZDEBGomicron-dev (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
146692025-04-30T06:13:37.352ZDEBGomicron-dev (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
146702025-04-30T06:13:37.352ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
146712025-04-30T06:13:37.369ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
146722025-04-30T06:13:37.370ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
146732025-04-30T06:13:37.370ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
146742025-04-30T06:13:37.370ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
146752025-04-30T06:13:37.370ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
146762025-04-30T06:13:37.370ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
146772025-04-30T06:13:37.370ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
146782025-04-30T06:13:37.371ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
146792025-04-30T06:13:37.371ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
146802025-04-30T06:13:37.375ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
146812025-04-30T06:13:37.376ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
146822025-04-30T06:13:37.376ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
146832025-04-30T06:13:37.381ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
146842025-04-30T06:13:37.381ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
146852025-04-30T06:13:37.381ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
146862025-04-30T06:13:37.431ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
146872025-04-30T06:13:37.431ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
146882025-04-30T06:13:37.431ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:49212 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
146892025-04-30T06:13:37.433ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:49212 req_id = 443b7491-d233-441e-bb6f-568450e1a774 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
146902025-04-30T06:13:37.434ZINFOomicron-dev (dropshot): request completed latency_us = 937 local_addr = [::1]:61881 method = GET remote_addr = [::1]:49212 req_id = 443b7491-d233-441e-bb6f-568450e1a774 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
146912025-04-30T06:13:37.436ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
146922025-04-30T06:13:37.439ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "443b7491-d233-441e-bb6f-568450e1a774", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
146932025-04-30T06:13:37.441ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
146942025-04-30T06:13:37.441ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
146952025-04-30T06:13:37.441ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
146962025-04-30T06:13:37.441ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
146972025-04-30T06:13:37.455ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
146982025-04-30T06:13:37.455ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
146992025-04-30T06:13:37.456ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147002025-04-30T06:13:37.461ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
147012025-04-30T06:13:37.461ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
147022025-04-30T06:13:37.461ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
147032025-04-30T06:13:37.477ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
147042025-04-30T06:13:37.479ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
147052025-04-30T06:13:37.480ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
147062025-04-30T06:13:37.485ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
147072025-04-30T06:13:37.485ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
147082025-04-30T06:13:37.487ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
147092025-04-30T06:13:37.487ZDEBGomicron-dev (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
147102025-04-30T06:13:37.488ZDEBGomicron-dev (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
147112025-04-30T06:13:37.488ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
147122025-04-30T06:13:37.489ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
147132025-04-30T06:13:37.491ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
147142025-04-30T06:13:37.493ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147152025-04-30T06:13:37.496ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
147162025-04-30T06:13:37.497ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
147172025-04-30T06:13:37.498ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
147182025-04-30T06:13:37.498ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
147192025-04-30T06:13:37.515ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
147202025-04-30T06:13:37.548ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
147212025-04-30T06:13:37.548ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
147222025-04-30T06:13:37.548ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147232025-04-30T06:13:37.548ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
147242025-04-30T06:13:37.549ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147252025-04-30T06:13:37.559ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147262025-04-30T06:13:37.565ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
147272025-04-30T06:13:37.565ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147282025-04-30T06:13:37.565ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
147292025-04-30T06:13:37.566ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147302025-04-30T06:13:37.587ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
147312025-04-30T06:13:37.588ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
147322025-04-30T06:13:37.588ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 896.529787ms iteration = 54 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
147332025-04-30T06:13:37.588ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 55 reason = Signaled
147342025-04-30T06:13:37.588ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
147352025-04-30T06:13:37.588ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
147362025-04-30T06:13:37.588ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
147372025-04-30T06:13:37.588ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
147382025-04-30T06:13:37.590ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147392025-04-30T06:13:37.590ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
147402025-04-30T06:13:37.590ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
147412025-04-30T06:13:37.591ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147422025-04-30T06:13:37.594ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147432025-04-30T06:13:37.598ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
147442025-04-30T06:13:37.598ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
147452025-04-30T06:13:37.598ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
147462025-04-30T06:13:37.598ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
147472025-04-30T06:13:37.602ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
147482025-04-30T06:13:37.602ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
147492025-04-30T06:13:37.603ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a5 5a 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .Z..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56641 req_id = 7f1f9b49-560f-4923-aacb-4815152ec26f
147502025-04-30T06:13:37.603ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42330,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56641 req_id = 7f1f9b49-560f-4923-aacb-4815152ec26f
147512025-04-30T06:13:37.603ZDEBGomicron-dev (store): query key key = _mgs._tcp
147522025-04-30T06:13:37.603ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
147532025-04-30T06:13:37.604ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56641 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 7f1f9b49-560f-4923-aacb-4815152ec26f
147542025-04-30T06:13:37.605ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2482, tv_nsec: 403225797 } })
147552025-04-30T06:13:37.605ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
147562025-04-30T06:13:37.605ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: fd 01 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58758 req_id = 99d89d32-7163-4ac0-8efa-fddc4be00e61
147572025-04-30T06:13:37.605ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64769,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58758 req_id = 99d89d32-7163-4ac0-8efa-fddc4be00e61
147582025-04-30T06:13:37.612ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
147592025-04-30T06:13:37.612ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58758 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 99d89d32-7163-4ac0-8efa-fddc4be00e61
147602025-04-30T06:13:37.622ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
147612025-04-30T06:13:37.666ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
147622025-04-30T06:13:37.668ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 1b e7 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61910 req_id = f66f543b-9889-41d9-81b7-9b0613cb132b
147632025-04-30T06:13:37.668ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 7143,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61910 req_id = f66f543b-9889-41d9-81b7-9b0613cb132b
147642025-04-30T06:13:37.668ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
147652025-04-30T06:13:37.668ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 7143, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61910 req_id = f66f543b-9889-41d9-81b7-9b0613cb132b
147662025-04-30T06:13:37.671ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
147672025-04-30T06:13:37.671ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
147682025-04-30T06:13:37.671ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
147692025-04-30T06:13:37.673ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
147702025-04-30T06:13:37.673ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
147712025-04-30T06:13:37.673ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147722025-04-30T06:13:37.673ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
147732025-04-30T06:13:37.674ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147742025-04-30T06:13:37.674ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffd7d439-ab22-46fc-84e9-90adfea6cc85", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
147752025-04-30T06:13:37.674ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
147762025-04-30T06:13:37.678ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
147772025-04-30T06:13:37.682ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
147782025-04-30T06:13:37.682ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147792025-04-30T06:13:37.682ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
147802025-04-30T06:13:37.683ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147812025-04-30T06:13:37.686ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
147822025-04-30T06:13:37.686ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e7108c8-53c9-4175-8dcc-e351435b7eb0", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
147832025-04-30T06:13:37.686ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
147842025-04-30T06:13:37.686ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
147852025-04-30T06:13:37.687ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
147862025-04-30T06:13:37.694ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "21f824bb-e6b9-4eea-a795-2279b37ec17a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
147872025-04-30T06:13:37.695ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
147882025-04-30T06:13:37.697ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0dc2c9f-b92f-4a4f-99d2-b75ae97295c2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
147892025-04-30T06:13:37.697ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
147902025-04-30T06:13:37.700ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "479405d3-1e37-4411-ad30-13cc82647400", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
147912025-04-30T06:13:37.701ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
147922025-04-30T06:13:37.704ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "728a45e4-4988-47b0-9618-e559f41585dc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
147932025-04-30T06:13:37.705ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
147942025-04-30T06:13:37.710ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5b909b3-2c09-43b1-b186-3dc1e63ee333", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
147952025-04-30T06:13:37.711ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
147962025-04-30T06:13:37.715ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
147972025-04-30T06:13:37.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
147982025-04-30T06:13:37.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
147992025-04-30T06:13:37.718ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6265f8ae-cd41-4561-9dff-6398c90e11f4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148002025-04-30T06:13:37.718ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
148012025-04-30T06:13:37.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
148022025-04-30T06:13:37.720ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
148032025-04-30T06:13:37.720ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
148042025-04-30T06:13:37.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
148052025-04-30T06:13:37.721ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "de36ab15-f567-4185-bc5f-ea0b47106304", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148062025-04-30T06:13:37.721ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
148072025-04-30T06:13:37.723ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a30eaf1b-5057-4e5f-b649-78861d581f7a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148082025-04-30T06:13:37.723ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
148092025-04-30T06:13:37.724ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f6a489ad-eb69-4354-8842-c516e0bfd3b9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148102025-04-30T06:13:37.724ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
148112025-04-30T06:13:37.727ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b870358f-7075-411f-b050-89b5e21b2b31", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148122025-04-30T06:13:37.727ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
148132025-04-30T06:13:37.729ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ade553c-90b8-406e-9880-f95e5a7dccad", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148142025-04-30T06:13:37.729ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
148152025-04-30T06:13:37.731ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a8a9af7e-fb8a-48fa-b3e0-605f77650378", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148162025-04-30T06:13:37.731ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
148172025-04-30T06:13:37.733ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "213cd53f-301e-4da0-b3a5-aec00df01a2b", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148182025-04-30T06:13:37.734ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
148192025-04-30T06:13:37.738ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc9970d2-215a-4bba-b8bc-c3f7f7171f1c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148202025-04-30T06:13:37.738ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
148212025-04-30T06:13:37.743ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "409f6060-b0e2-418b-bb98-b15bdcf0cbca", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148222025-04-30T06:13:37.743ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
148232025-04-30T06:13:37.748ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca5f63c8-20df-4b27-8b50-cb4669c966d8", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148242025-04-30T06:13:37.748ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
148252025-04-30T06:13:37.751ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c6e1ae1-00e6-40f1-989f-7c90c7aa888a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148262025-04-30T06:13:37.751ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
148272025-04-30T06:13:37.753ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7250870c-cc40-42f8-90d1-14f4ee4c13a9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148282025-04-30T06:13:37.754ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
148292025-04-30T06:13:37.760ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8db8dfef-510f-495f-936f-61c71a8a2d96", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148302025-04-30T06:13:37.761ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
148312025-04-30T06:13:37.765ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e810c6aa-cc59-4ea5-9640-c4135c7714f4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148322025-04-30T06:13:37.766ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
148332025-04-30T06:13:37.766ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
148342025-04-30T06:13:37.767ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
148352025-04-30T06:13:37.767ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
148362025-04-30T06:13:37.767ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
148372025-04-30T06:13:37.767ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
148382025-04-30T06:13:37.770ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f75b8714-519c-4026-90c8-f173af97e89a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148392025-04-30T06:13:37.770ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
148402025-04-30T06:13:37.771ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148412025-04-30T06:13:37.774ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "813233ef-add9-482f-a380-2205b10ada7a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148422025-04-30T06:13:37.774ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
148432025-04-30T06:13:37.776ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
148442025-04-30T06:13:37.776ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
148452025-04-30T06:13:37.776ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
148462025-04-30T06:13:37.776ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
148472025-04-30T06:13:37.779ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
148482025-04-30T06:13:37.779ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
148492025-04-30T06:13:37.779ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
148502025-04-30T06:13:37.780ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "44292a27-d38f-4527-8c3a-daa2120b599a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148512025-04-30T06:13:37.780ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
148522025-04-30T06:13:37.783ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3035a64-ac76-4e2e-a935-51383ad7e8e9", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148532025-04-30T06:13:37.783ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
148542025-04-30T06:13:37.786ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "24a1959a-7daa-4a99-b598-ef12a4111545", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148552025-04-30T06:13:37.786ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
148562025-04-30T06:13:37.789ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "161271fa-2199-4214-85c8-4b5ddfe59027", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148572025-04-30T06:13:37.789ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
148582025-04-30T06:13:37.790ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eec8015e-a455-43d2-baa5-8cbceee2d295", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148592025-04-30T06:13:37.790ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
148602025-04-30T06:13:37.795ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b06d5a0-d490-4b88-91c1-f131c55a8d89", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148612025-04-30T06:13:37.795ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
148622025-04-30T06:13:37.796ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a52a4a8f-d5e7-4b19-9fda-f8646bcc3603", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148632025-04-30T06:13:37.797ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
148642025-04-30T06:13:37.798ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f27b8bac-0d86-42e7-9bfc-f78d242a868b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148652025-04-30T06:13:37.798ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
148662025-04-30T06:13:37.801ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bed8a44b-cad0-4cd6-9b2a-c366d84218ab", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148672025-04-30T06:13:37.802ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
148682025-04-30T06:13:37.804ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c07fe721-a64f-4c1e-b50f-757619573890", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148692025-04-30T06:13:37.805ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
148702025-04-30T06:13:37.807ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "64fce516-ae3b-4f52-8903-00e2622a59bb", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148712025-04-30T06:13:37.808ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
148722025-04-30T06:13:37.809ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
148732025-04-30T06:13:37.816ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "727f99ab-d52f-4abd-90b5-c2eaa1503145", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148742025-04-30T06:13:37.816ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
148752025-04-30T06:13:37.820ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1dade6e0-b03d-49b7-b471-384a7ffbb44c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148762025-04-30T06:13:37.821ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
148772025-04-30T06:13:37.825ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7122b8b3-692a-4261-aec7-3627ad7ad834", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148782025-04-30T06:13:37.825ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
148792025-04-30T06:13:37.827ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4eb9a68d-088d-4d96-a340-98850241db18", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148802025-04-30T06:13:37.827ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
148812025-04-30T06:13:37.830ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c65fd814-6bb0-4c0d-8d71-4af335edb0e1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148822025-04-30T06:13:37.830ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
148832025-04-30T06:13:37.834ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c4b2ed4d-b5ad-4d7f-bb19-4e3f9c9beaa3", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148842025-04-30T06:13:37.835ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
148852025-04-30T06:13:37.836ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "374c55be-70a3-4488-9362-f64a24aa3f74", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:36 GMT"} })
148862025-04-30T06:13:37.836ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
148872025-04-30T06:13:37.837ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca5f9cb9-70c7-4ab2-85f6-f7c295ec3028", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148882025-04-30T06:13:37.838ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
148892025-04-30T06:13:37.840ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6805a78d-e0db-4424-b394-6c56074e79f9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148902025-04-30T06:13:37.840ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
148912025-04-30T06:13:37.843ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a39dfba-c034-48e9-a389-88de374da8e6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
148922025-04-30T06:13:37.843ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
148932025-04-30T06:13:37.843ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
148942025-04-30T06:13:37.843ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
148952025-04-30T06:13:37.843ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
148962025-04-30T06:13:37.844ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
148972025-04-30T06:13:37.848ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
148982025-04-30T06:13:37.853ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
148992025-04-30T06:13:37.853ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
149002025-04-30T06:13:37.853ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
149012025-04-30T06:13:37.853ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149022025-04-30T06:13:37.853ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149032025-04-30T06:13:37.854ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149042025-04-30T06:13:37.858ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149052025-04-30T06:13:37.861ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
149062025-04-30T06:13:37.861ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
149072025-04-30T06:13:37.861ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149082025-04-30T06:13:37.862ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
149092025-04-30T06:13:37.862ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149102025-04-30T06:13:37.869ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
149112025-04-30T06:13:37.870ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149122025-04-30T06:13:37.870ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149132025-04-30T06:13:37.872ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149142025-04-30T06:13:37.877ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149152025-04-30T06:13:37.882ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
149162025-04-30T06:13:37.882ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149172025-04-30T06:13:37.882ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
149182025-04-30T06:13:37.882ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149192025-04-30T06:13:37.980ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
149202025-04-30T06:13:37.980ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
149212025-04-30T06:13:37.981ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:53370 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
149222025-04-30T06:13:37.982ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:53370 req_id = d3ec7c02-61ca-4b08-a8ab-03546ca59798 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
149232025-04-30T06:13:37.983ZINFOomicron-dev (dropshot): request completed latency_us = 758 local_addr = [::1]:61881 method = GET remote_addr = [::1]:53370 req_id = d3ec7c02-61ca-4b08-a8ab-03546ca59798 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
149242025-04-30T06:13:37.984ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "d3ec7c02-61ca-4b08-a8ab-03546ca59798", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
149252025-04-30T06:13:37.985ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149262025-04-30T06:13:37.985ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
149272025-04-30T06:13:37.987ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149282025-04-30T06:13:37.989ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
149292025-04-30T06:13:37.989ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
149302025-04-30T06:13:37.989ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
149312025-04-30T06:13:37.992ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
149322025-04-30T06:13:37.992ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
149332025-04-30T06:13:37.992ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149342025-04-30T06:13:37.992ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149352025-04-30T06:13:37.995ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149362025-04-30T06:13:38.008ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149372025-04-30T06:13:38.013ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
149382025-04-30T06:13:38.014ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149392025-04-30T06:13:38.014ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149402025-04-30T06:13:38.014ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149412025-04-30T06:13:38.029ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
149422025-04-30T06:13:38.086ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
149432025-04-30T06:13:38.086ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
149442025-04-30T06:13:38.087ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149452025-04-30T06:13:38.087ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
149462025-04-30T06:13:38.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
149472025-04-30T06:13:38.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
149482025-04-30T06:13:38.091ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149492025-04-30T06:13:38.091ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
149502025-04-30T06:13:38.091ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
149512025-04-30T06:13:38.091ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
149522025-04-30T06:13:38.092ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
149532025-04-30T06:13:38.092ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 504.538841ms iteration = 55 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
149542025-04-30T06:13:38.093ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 56 reason = Signaled
149552025-04-30T06:13:38.093ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
149562025-04-30T06:13:38.093ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
149572025-04-30T06:13:38.093ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149582025-04-30T06:13:38.093ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149592025-04-30T06:13:38.095ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149602025-04-30T06:13:38.095ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
149612025-04-30T06:13:38.096ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
149622025-04-30T06:13:38.101ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
149632025-04-30T06:13:38.101ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
149642025-04-30T06:13:38.103ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149652025-04-30T06:13:38.105ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
149662025-04-30T06:13:38.110ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
149672025-04-30T06:13:38.110ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
149682025-04-30T06:13:38.110ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
149692025-04-30T06:13:38.110ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
149702025-04-30T06:13:38.112ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
149712025-04-30T06:13:38.112ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149722025-04-30T06:13:38.112ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
149732025-04-30T06:13:38.113ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149742025-04-30T06:13:38.122ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
149752025-04-30T06:13:38.122ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
149762025-04-30T06:13:38.122ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
149772025-04-30T06:13:38.124ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
149782025-04-30T06:13:38.124ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
149792025-04-30T06:13:38.125ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 9a 3f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .?..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:61979 req_id = 1c2f330b-e03f-4204-b999-c589ba85ddc8
149802025-04-30T06:13:38.125ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 39487,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61979 req_id = 1c2f330b-e03f-4204-b999-c589ba85ddc8
149812025-04-30T06:13:38.125ZDEBGomicron-dev (store): query key key = _mgs._tcp
149822025-04-30T06:13:38.125ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
149832025-04-30T06:13:38.126ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:61979 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1c2f330b-e03f-4204-b999-c589ba85ddc8
149842025-04-30T06:13:38.127ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2482, tv_nsec: 926093347 } })
149852025-04-30T06:13:38.127ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
149862025-04-30T06:13:38.127ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ea 0a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54989 req_id = e12a6cb2-e145-463b-b899-47b6816cda56
149872025-04-30T06:13:38.127ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 59914,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54989 req_id = e12a6cb2-e145-463b-b899-47b6816cda56
149882025-04-30T06:13:38.128ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
149892025-04-30T06:13:38.128ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54989 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = e12a6cb2-e145-463b-b899-47b6816cda56
149902025-04-30T06:13:38.164ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
149912025-04-30T06:13:38.179ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
149922025-04-30T06:13:38.180ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 74 71 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c tq..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:52450 req_id = 50bc58c2-3c0a-4c93-af22-7c2f2f406654
149932025-04-30T06:13:38.180ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29809,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52450 req_id = 50bc58c2-3c0a-4c93-af22-7c2f2f406654
149942025-04-30T06:13:38.181ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
149952025-04-30T06:13:38.181ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 29809, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:52450 req_id = 50bc58c2-3c0a-4c93-af22-7c2f2f406654
149962025-04-30T06:13:38.181ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
149972025-04-30T06:13:38.181ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
149982025-04-30T06:13:38.181ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
149992025-04-30T06:13:38.183ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad106b60-baea-4dc9-b509-01664427b91e", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150002025-04-30T06:13:38.184ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
150012025-04-30T06:13:38.185ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9162dcd-06b7-41ed-ae20-7fcc5bcdb2a1", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150022025-04-30T06:13:38.185ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
150032025-04-30T06:13:38.189ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "792bdbaa-f765-43d3-bf81-9a052be0dc87", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150042025-04-30T06:13:38.189ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
150052025-04-30T06:13:38.192ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
150062025-04-30T06:13:38.193ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cdec490b-e0ea-4df5-b368-9eda6d6afda0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150072025-04-30T06:13:38.193ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
150082025-04-30T06:13:38.195ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "56f30c19-ed9a-4487-82fd-de50252cd551", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150092025-04-30T06:13:38.196ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
150102025-04-30T06:13:38.198ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "04e38a24-43a3-4c6e-9dab-29500c767485", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150112025-04-30T06:13:38.198ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
150122025-04-30T06:13:38.200ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3218899f-8517-4db5-860e-289c41a0b8f1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150132025-04-30T06:13:38.200ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
150142025-04-30T06:13:38.208ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4357d7b0-b896-481a-af2b-e34ec3362ed1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150152025-04-30T06:13:38.208ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
150162025-04-30T06:13:38.211ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2d0e90f4-6f59-41b3-898c-7a95d3f17105", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150172025-04-30T06:13:38.211ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
150182025-04-30T06:13:38.213ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6c4de1f-51d7-4c56-aa69-679df627740e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150192025-04-30T06:13:38.214ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
150202025-04-30T06:13:38.215ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
150212025-04-30T06:13:38.215ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
150222025-04-30T06:13:38.215ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
150232025-04-30T06:13:38.215ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
150242025-04-30T06:13:38.216ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
150252025-04-30T06:13:38.216ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "81c2bbf3-7281-4399-90af-3eec10d0aadb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150262025-04-30T06:13:38.216ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
150272025-04-30T06:13:38.218ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
150282025-04-30T06:13:38.218ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "26d43712-bdf9-4742-a44c-e3b341febceb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150292025-04-30T06:13:38.219ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
150302025-04-30T06:13:38.221ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b070047-5afe-42af-bfa8-5d69cfb9dd35", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150312025-04-30T06:13:38.221ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
150322025-04-30T06:13:38.225ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7e5b09c5-1460-462b-8100-9a190ee0b15d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150332025-04-30T06:13:38.225ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
150342025-04-30T06:13:38.226ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
150352025-04-30T06:13:38.226ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
150362025-04-30T06:13:38.227ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "80f6b3fa-1336-41c5-bb39-243760033e81", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150372025-04-30T06:13:38.227ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
150382025-04-30T06:13:38.229ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4463c95f-8d07-499b-ab7e-6ebf4bf13957", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150392025-04-30T06:13:38.229ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
150402025-04-30T06:13:38.230ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
150412025-04-30T06:13:38.232ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
150422025-04-30T06:13:38.233ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b3357da1-dcd1-4fda-b29a-01a3a8b638c0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150432025-04-30T06:13:38.234ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
150442025-04-30T06:13:38.235ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
150452025-04-30T06:13:38.236ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
150462025-04-30T06:13:38.236ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "167612cc-cb2c-463c-aa1b-571b6cc5f2bc", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150472025-04-30T06:13:38.236ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
150482025-04-30T06:13:38.236ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
150492025-04-30T06:13:38.239ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c101b1c-26a7-4632-8d77-1a4e6b07e093", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150502025-04-30T06:13:38.240ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
150512025-04-30T06:13:38.241ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8b85e85f-46af-4f02-b01e-70dd6629873f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150522025-04-30T06:13:38.241ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
150532025-04-30T06:13:38.242ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "57736d10-a851-4066-85f7-6e923c333937", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150542025-04-30T06:13:38.242ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
150552025-04-30T06:13:38.244ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9a477e1-bebc-4e1a-8edb-428300a3c7f5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150562025-04-30T06:13:38.245ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
150572025-04-30T06:13:38.247ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a30b432d-f2ff-4a3d-9d4d-b5e830e7f0c6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150582025-04-30T06:13:38.248ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
150592025-04-30T06:13:38.250ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b7d69cb7-444b-4d6f-b773-182c3f31945b", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150602025-04-30T06:13:38.250ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
150612025-04-30T06:13:38.253ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1bb5f43-3a0c-4d55-b2f8-930524810c34", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150622025-04-30T06:13:38.253ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
150632025-04-30T06:13:38.256ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2f970ff-b452-49cf-a4c1-a241692a3905", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150642025-04-30T06:13:38.256ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
150652025-04-30T06:13:38.260ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0ce1aeff-642b-4377-80ed-c9382d5ecae3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150662025-04-30T06:13:38.260ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
150672025-04-30T06:13:38.262ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a814eac-48a0-4ee4-ad35-283002a41de2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150682025-04-30T06:13:38.262ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
150692025-04-30T06:13:38.264ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "227c6641-8b96-48d3-985a-fc24de957d04", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150702025-04-30T06:13:38.264ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
150712025-04-30T06:13:38.268ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ffcec794-8e5b-47fd-9cb2-7de7e245c1d4", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150722025-04-30T06:13:38.268ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
150732025-04-30T06:13:38.269ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "63b64b0e-62bb-41c3-a847-a5339b0b4dda", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
150742025-04-30T06:13:38.269ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
150752025-04-30T06:13:38.270ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "01c7a4d7-96b5-47cc-8649-0bbfaf2d4661", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150762025-04-30T06:13:38.270ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
150772025-04-30T06:13:38.272ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "57d2b660-307b-4238-ac2f-602e48415e5c", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150782025-04-30T06:13:38.272ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
150792025-04-30T06:13:38.273ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "41005d48-de24-4af1-b382-e0c8b8a6ce63", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150802025-04-30T06:13:38.273ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
150812025-04-30T06:13:38.275ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b16c5f7-3575-4622-8304-28b5b7e05fdc", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150822025-04-30T06:13:38.275ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
150832025-04-30T06:13:38.278ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
150842025-04-30T06:13:38.278ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1213d37-3cef-4d85-bc14-1a2119144c4a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150852025-04-30T06:13:38.279ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
150862025-04-30T06:13:38.281ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f22cec03-1165-4299-9034-42be2bf8d71c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150872025-04-30T06:13:38.281ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
150882025-04-30T06:13:38.283ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0164389c-14e5-45c5-a25f-7a38ff2788b8", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150892025-04-30T06:13:38.283ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
150902025-04-30T06:13:38.285ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6cc7ad15-fcb4-4b6c-9d91-2da0688121a6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150912025-04-30T06:13:38.285ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
150922025-04-30T06:13:38.287ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b42395d-1182-42c6-a5d2-7752d9f42415", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150932025-04-30T06:13:38.288ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
150942025-04-30T06:13:38.289ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "96eace6e-5a4c-484d-b91a-63fc0ac49ec1", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150952025-04-30T06:13:38.289ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
150962025-04-30T06:13:38.290ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "77b26bfb-b12d-4bef-b700-b90206980ae9", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150972025-04-30T06:13:38.290ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
150982025-04-30T06:13:38.292ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4421d6a-fe85-4ae5-82dc-8d8f8f364a73", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
150992025-04-30T06:13:38.292ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
151002025-04-30T06:13:38.294ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7ab32b30-b070-4772-b493-e1767217397b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
151012025-04-30T06:13:38.294ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
151022025-04-30T06:13:38.296ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "73643243-6746-4d90-bf8c-37a0228b379f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
151032025-04-30T06:13:38.296ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
151042025-04-30T06:13:38.296ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151052025-04-30T06:13:38.297ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151062025-04-30T06:13:38.297ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151072025-04-30T06:13:38.298ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151082025-04-30T06:13:38.302ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151092025-04-30T06:13:38.306ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
151102025-04-30T06:13:38.306ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
151112025-04-30T06:13:38.306ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151122025-04-30T06:13:38.306ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151132025-04-30T06:13:38.306ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151142025-04-30T06:13:38.307ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151152025-04-30T06:13:38.311ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151162025-04-30T06:13:38.315ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
151172025-04-30T06:13:38.315ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151182025-04-30T06:13:38.315ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151192025-04-30T06:13:38.316ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151202025-04-30T06:13:38.330ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
151212025-04-30T06:13:38.330ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151222025-04-30T06:13:38.330ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
151232025-04-30T06:13:38.330ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
151242025-04-30T06:13:38.331ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
151252025-04-30T06:13:38.379ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
151262025-04-30T06:13:38.380ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
151272025-04-30T06:13:38.380ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:53422 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
151282025-04-30T06:13:38.381ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:53422 req_id = 21cbadbf-aba9-48cc-b3b6-864cac8bc641 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
151292025-04-30T06:13:38.382ZINFOomicron-dev (dropshot): request completed latency_us = 660 local_addr = [::1]:61881 method = GET remote_addr = [::1]:53422 req_id = 21cbadbf-aba9-48cc-b3b6-864cac8bc641 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
151302025-04-30T06:13:38.382ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "21cbadbf-aba9-48cc-b3b6-864cac8bc641", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:37 GMT"} })
151312025-04-30T06:13:38.385ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
151322025-04-30T06:13:38.385ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
151332025-04-30T06:13:38.386ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
151342025-04-30T06:13:38.388ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
151352025-04-30T06:13:38.388ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151362025-04-30T06:13:38.389ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151372025-04-30T06:13:38.389ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151382025-04-30T06:13:38.390ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151392025-04-30T06:13:38.391ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151402025-04-30T06:13:38.393ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151412025-04-30T06:13:38.397ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
151422025-04-30T06:13:38.398ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151432025-04-30T06:13:38.398ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151442025-04-30T06:13:38.398ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151452025-04-30T06:13:38.401ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
151462025-04-30T06:13:38.401ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
151472025-04-30T06:13:38.401ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
151482025-04-30T06:13:38.402ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
151492025-04-30T06:13:38.405ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
151502025-04-30T06:13:38.405ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
151512025-04-30T06:13:38.406ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
151522025-04-30T06:13:38.430ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
151532025-04-30T06:13:38.431ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
151542025-04-30T06:13:38.431ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 338.488366ms iteration = 56 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
151552025-04-30T06:13:38.431ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 57 reason = Signaled
151562025-04-30T06:13:38.431ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
151572025-04-30T06:13:38.432ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151582025-04-30T06:13:38.432ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151592025-04-30T06:13:38.432ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151602025-04-30T06:13:38.436ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151612025-04-30T06:13:38.441ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151622025-04-30T06:13:38.446ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
151632025-04-30T06:13:38.450ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
151642025-04-30T06:13:38.450ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
151652025-04-30T06:13:38.450ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
151662025-04-30T06:13:38.450ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
151672025-04-30T06:13:38.454ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
151682025-04-30T06:13:38.454ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
151692025-04-30T06:13:38.454ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 14 9d 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:50825 req_id = 4f1121e4-834b-464c-8357-2f881fd2aefd
151702025-04-30T06:13:38.455ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5277,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50825 req_id = 4f1121e4-834b-464c-8357-2f881fd2aefd
151712025-04-30T06:13:38.455ZDEBGomicron-dev (store): query key key = _mgs._tcp
151722025-04-30T06:13:38.455ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
151732025-04-30T06:13:38.455ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:50825 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 4f1121e4-834b-464c-8357-2f881fd2aefd
151742025-04-30T06:13:38.455ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2483, tv_nsec: 254797088 } })
151752025-04-30T06:13:38.455ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
151762025-04-30T06:13:38.456ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 5d 6a 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ]j..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50636 req_id = 1fa0ea60-f162-413d-b338-0a6feaa8f217
151772025-04-30T06:13:38.456ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23914,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50636 req_id = 1fa0ea60-f162-413d-b338-0a6feaa8f217
151782025-04-30T06:13:38.456ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
151792025-04-30T06:13:38.456ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50636 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1fa0ea60-f162-413d-b338-0a6feaa8f217
151802025-04-30T06:13:38.501ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
151812025-04-30T06:13:38.502ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 5d 2b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ]+..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:51445 req_id = 6cb24bab-15e5-40b4-a8cc-658bd996cc59
151822025-04-30T06:13:38.502ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
151832025-04-30T06:13:38.502ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23851,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51445 req_id = 6cb24bab-15e5-40b4-a8cc-658bd996cc59
151842025-04-30T06:13:38.502ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
151852025-04-30T06:13:38.502ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
151862025-04-30T06:13:38.502ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
151872025-04-30T06:13:38.502ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
151882025-04-30T06:13:38.503ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 23851, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:51445 req_id = 6cb24bab-15e5-40b4-a8cc-658bd996cc59
151892025-04-30T06:13:38.503ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
151902025-04-30T06:13:38.503ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
151912025-04-30T06:13:38.503ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
151922025-04-30T06:13:38.504ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
151932025-04-30T06:13:38.506ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "09c162d5-caf8-4c9a-8475-4aea77b874f1", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
151942025-04-30T06:13:38.507ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
151952025-04-30T06:13:38.507ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
151962025-04-30T06:13:38.509ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8a022ae-9294-4b41-bab6-37850f71220c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
151972025-04-30T06:13:38.510ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
151982025-04-30T06:13:38.512ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca5ac642-37ce-488d-9a82-e3f3e4432831", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
151992025-04-30T06:13:38.512ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
152002025-04-30T06:13:38.514ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "70c26c08-12b5-43ef-ae35-6573668e00e1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152012025-04-30T06:13:38.514ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
152022025-04-30T06:13:38.516ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfd8ba38-69e0-4821-a26a-62c9a6a2f3d7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152032025-04-30T06:13:38.516ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
152042025-04-30T06:13:38.518ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "08d28b31-960d-4f90-a2ee-2343a811a9ff", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152052025-04-30T06:13:38.518ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
152062025-04-30T06:13:38.519ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
152072025-04-30T06:13:38.519ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
152082025-04-30T06:13:38.519ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
152092025-04-30T06:13:38.520ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7722ef88-c571-4efa-954e-268606af363f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152102025-04-30T06:13:38.520ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
152112025-04-30T06:13:38.521ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
152122025-04-30T06:13:38.522ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9630d9a-2d91-4b6d-9dcb-f42ed9d70956", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152132025-04-30T06:13:38.522ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
152142025-04-30T06:13:38.524ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
152152025-04-30T06:13:38.524ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
152162025-04-30T06:13:38.525ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "016663c4-cd8b-471e-b5ac-c56b788955d0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152172025-04-30T06:13:38.525ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
152182025-04-30T06:13:38.526ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
152192025-04-30T06:13:38.527ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc0cb8ec-4a8c-4bc4-964c-20e860ce316e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152202025-04-30T06:13:38.528ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
152212025-04-30T06:13:38.531ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d4e753c1-f54a-4bce-be98-db3e4830bcc3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152222025-04-30T06:13:38.532ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
152232025-04-30T06:13:38.534ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "52649372-2dc0-4741-8db5-8ab60fb9f8eb", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152242025-04-30T06:13:38.534ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
152252025-04-30T06:13:38.537ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9e53cf03-388e-40a2-9e6a-72813e9ec83d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152262025-04-30T06:13:38.538ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
152272025-04-30T06:13:38.541ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "205f5d7a-68a4-460a-b6e3-a461433f99cb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152282025-04-30T06:13:38.541ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
152292025-04-30T06:13:38.544ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c855b8a5-3390-4c99-980a-3c544ec691ba", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152302025-04-30T06:13:38.544ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
152312025-04-30T06:13:38.549ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f26077dd-9fe9-435f-ae60-a443fa795efa", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152322025-04-30T06:13:38.549ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
152332025-04-30T06:13:38.566ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:32779 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
152342025-04-30T06:13:38.568ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:32779 req_id = ea559972-f5a9-4e6c-9569-2792626e7494 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
152352025-04-30T06:13:38.568ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ec947ac3-a97d-4a76-83e2-38903a24f3af", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152362025-04-30T06:13:38.569ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
152372025-04-30T06:13:38.570ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 2625 local_addr = [::1]:55020 method = GET remote_addr = [::1]:32779 req_id = ea559972-f5a9-4e6c-9569-2792626e7494 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
152382025-04-30T06:13:38.573ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
152392025-04-30T06:13:38.575ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d6256411-3750-4ca4-af09-a0730802d7f1", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152402025-04-30T06:13:38.576ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
152412025-04-30T06:13:38.579ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "470a5390-ed18-4551-a3e5-6a96734338b2", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152422025-04-30T06:13:38.579ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
152432025-04-30T06:13:38.581ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "726ab43e-6af0-4354-b4fe-a899548b209c", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152442025-04-30T06:13:38.581ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
152452025-04-30T06:13:38.584ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0fe2f784-fdb4-4349-86d4-a1f8e62197a1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152462025-04-30T06:13:38.584ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
152472025-04-30T06:13:38.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "385b2513-744b-44a0-a359-d9f02fbb6915", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152482025-04-30T06:13:38.586ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
152492025-04-30T06:13:38.587ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43b05515-bfdd-4565-9def-228e77a74bd9", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152502025-04-30T06:13:38.587ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
152512025-04-30T06:13:38.594ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8145ea67-36df-497b-ad0b-e9169e2f8130", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152522025-04-30T06:13:38.594ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
152532025-04-30T06:13:38.594ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d64006e7-20ae-48e0-b036-138cafdb6c00", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152542025-04-30T06:13:38.594ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
152552025-04-30T06:13:38.600ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a9249f9-f7b0-4260-9f06-b934697c2ab6", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152562025-04-30T06:13:38.600ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
152572025-04-30T06:13:38.601ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a9be3db-93a3-446c-86c2-6fd0f44a49d7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152582025-04-30T06:13:38.608ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
152592025-04-30T06:13:38.608ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "46b5734e-adb4-4708-bf9f-587eebe5d1c9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152602025-04-30T06:13:38.608ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
152612025-04-30T06:13:38.608ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "749fc86f-e0b2-456c-b4fa-ebf52d7bc05d", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152622025-04-30T06:13:38.608ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
152632025-04-30T06:13:38.608ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a259960a-d0ea-4fed-b404-90d0442056cf", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152642025-04-30T06:13:38.608ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
152652025-04-30T06:13:38.609ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "647ddd6d-09f6-4c06-a686-b31cbdc89c9f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152662025-04-30T06:13:38.609ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
152672025-04-30T06:13:38.614ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "406142e1-ef2a-4620-b700-6adb8ddd9744", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152682025-04-30T06:13:38.614ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
152692025-04-30T06:13:38.617ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0459e49a-22fa-4e72-a859-e5c09df7e9b6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152702025-04-30T06:13:38.617ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
152712025-04-30T06:13:38.619ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f6e2b93-81e6-4de0-a075-cf3a0c02a548", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152722025-04-30T06:13:38.619ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
152732025-04-30T06:13:38.620ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5034721a-f83b-4944-a472-3c4c975de3cb", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152742025-04-30T06:13:38.621ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
152752025-04-30T06:13:38.623ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ec085df-ace2-48cb-8bf7-50cef3e0417e", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152762025-04-30T06:13:38.623ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
152772025-04-30T06:13:38.624ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
152782025-04-30T06:13:38.624ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
152792025-04-30T06:13:38.624ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
152802025-04-30T06:13:38.624ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
152812025-04-30T06:13:38.625ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
152822025-04-30T06:13:38.625ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fbb9bef-e3d5-4aa2-99c2-84d0317905f1", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152832025-04-30T06:13:38.625ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
152842025-04-30T06:13:38.628ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
152852025-04-30T06:13:38.630ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "72f7ea2d-30e8-455c-a51a-987570be838c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152862025-04-30T06:13:38.630ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
152872025-04-30T06:13:38.633ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "50c2bcc2-d328-4865-84ff-0908f382454f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152882025-04-30T06:13:38.633ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
152892025-04-30T06:13:38.634ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
152902025-04-30T06:13:38.634ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
152912025-04-30T06:13:38.634ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
152922025-04-30T06:13:38.635ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
152932025-04-30T06:13:38.636ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f804fd37-8550-4c84-be24-2c213bfb94ab", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152942025-04-30T06:13:38.637ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
152952025-04-30T06:13:38.639ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
152962025-04-30T06:13:38.639ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
152972025-04-30T06:13:38.640ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b313287-2968-448c-a3a8-7e929e6720b8", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
152982025-04-30T06:13:38.640ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
152992025-04-30T06:13:38.640ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
153002025-04-30T06:13:38.641ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7683a43-25eb-4687-b737-f05d64ca44e8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
153012025-04-30T06:13:38.641ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
153022025-04-30T06:13:38.643ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "33aba5dc-5d9a-4dfb-9876-7bd1ce3b2bd9", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
153032025-04-30T06:13:38.643ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
153042025-04-30T06:13:38.644ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6dd569d6-c40e-4cb7-84aa-3377ad23f1b6", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
153052025-04-30T06:13:38.644ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
153062025-04-30T06:13:38.646ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8494c861-1c10-4f05-b8e3-bf14d78f5abd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
153072025-04-30T06:13:38.646ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
153082025-04-30T06:13:38.646ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
153092025-04-30T06:13:38.646ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153102025-04-30T06:13:38.646ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153112025-04-30T06:13:38.647ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153122025-04-30T06:13:38.650ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153132025-04-30T06:13:38.655ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
153142025-04-30T06:13:38.655ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
153152025-04-30T06:13:38.655ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
153162025-04-30T06:13:38.655ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153172025-04-30T06:13:38.655ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153182025-04-30T06:13:38.656ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153192025-04-30T06:13:38.660ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153202025-04-30T06:13:38.669ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
153212025-04-30T06:13:38.670ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153222025-04-30T06:13:38.671ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153232025-04-30T06:13:38.673ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153242025-04-30T06:13:38.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
153252025-04-30T06:13:38.680ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153262025-04-30T06:13:38.682ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0c88bc62-c00a-4bf3-aa4c-75ef2d5b3174 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153272025-04-30T06:13:38.682ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0c88bc62-c00a-4bf3-aa4c-75ef2d5b3174 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153282025-04-30T06:13:38.682ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0c88bc62-c00a-4bf3-aa4c-75ef2d5b3174 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153292025-04-30T06:13:38.682ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0c88bc62-c00a-4bf3-aa4c-75ef2d5b3174 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153302025-04-30T06:13:38.697ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
153312025-04-30T06:13:38.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
153322025-04-30T06:13:38.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
153332025-04-30T06:13:38.722ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
153342025-04-30T06:13:38.723ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
153352025-04-30T06:13:38.723ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
153362025-04-30T06:13:38.723ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
153372025-04-30T06:13:38.729ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
153382025-04-30T06:13:38.729ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
153392025-04-30T06:13:38.730ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:38200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
153402025-04-30T06:13:38.731ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:38200 req_id = 7f298960-61d5-4871-b65d-c52f8bd8eac3 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
153412025-04-30T06:13:38.732ZINFOomicron-dev (dropshot): request completed latency_us = 579 local_addr = [::1]:61881 method = GET remote_addr = [::1]:38200 req_id = 7f298960-61d5-4871-b65d-c52f8bd8eac3 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
153422025-04-30T06:13:38.732ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7f298960-61d5-4871-b65d-c52f8bd8eac3", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
153432025-04-30T06:13:38.733ZINFOomicron-dev (dropshot_internal): request completed latency_us = 52372 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 0c88bc62-c00a-4bf3-aa4c-75ef2d5b3174 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
153442025-04-30T06:13:38.735ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c88bc62-c00a-4bf3-aa4c-75ef2d5b3174", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
153452025-04-30T06:13:38.735ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
153462025-04-30T06:13:38.735ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 21e23869-a16d-4a29-9617-24dfd66740ef uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
153472025-04-30T06:13:38.736ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
153482025-04-30T06:13:38.736ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
153492025-04-30T06:13:38.736ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
153502025-04-30T06:13:38.736ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 21e23869-a16d-4a29-9617-24dfd66740ef resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
153512025-04-30T06:13:38.736ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 21e23869-a16d-4a29-9617-24dfd66740ef roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
153522025-04-30T06:13:38.737ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 21e23869-a16d-4a29-9617-24dfd66740ef resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
153532025-04-30T06:13:38.739ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
153542025-04-30T06:13:38.739ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
153552025-04-30T06:13:38.739ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153562025-04-30T06:13:38.739ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153572025-04-30T06:13:38.740ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153582025-04-30T06:13:38.740ZINFOomicron-dev (dropshot_internal): request completed latency_us = 4838 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 21e23869-a16d-4a29-9617-24dfd66740ef response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
153592025-04-30T06:13:38.741ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "21e23869-a16d-4a29-9617-24dfd66740ef", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
153602025-04-30T06:13:38.741ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
153612025-04-30T06:13:38.741ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
153622025-04-30T06:13:38.741ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
153632025-04-30T06:13:38.743ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153642025-04-30T06:13:38.751ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
153652025-04-30T06:13:38.751ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153662025-04-30T06:13:38.751ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153672025-04-30T06:13:38.752ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153682025-04-30T06:13:38.756ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
153692025-04-30T06:13:38.757ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
153702025-04-30T06:13:38.757ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
153712025-04-30T06:13:38.757ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
153722025-04-30T06:13:38.760ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
153732025-04-30T06:13:38.765ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153742025-04-30T06:13:38.775ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
153752025-04-30T06:13:38.775ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
153762025-04-30T06:13:38.775ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
153772025-04-30T06:13:38.776ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
153782025-04-30T06:13:38.791ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
153792025-04-30T06:13:38.793ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
153802025-04-30T06:13:38.794ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 362.266752ms iteration = 57 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
153812025-04-30T06:13:38.794ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 58 reason = Signaled
153822025-04-30T06:13:38.794ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
153832025-04-30T06:13:38.794ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
153842025-04-30T06:13:38.794ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153852025-04-30T06:13:38.794ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153862025-04-30T06:13:38.797ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153872025-04-30T06:13:38.798ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
153882025-04-30T06:13:38.798ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
153892025-04-30T06:13:38.800ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
153902025-04-30T06:13:38.801ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
153912025-04-30T06:13:38.807ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
153922025-04-30T06:13:38.807ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
153932025-04-30T06:13:38.807ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
153942025-04-30T06:13:38.808ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
153952025-04-30T06:13:38.813ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
153962025-04-30T06:13:38.813ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
153972025-04-30T06:13:38.813ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: df 7f 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:59195 req_id = 8ef382d5-f10f-4cc9-93f2-a6596b179a52
153982025-04-30T06:13:38.813ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57215,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59195 req_id = 8ef382d5-f10f-4cc9-93f2-a6596b179a52
153992025-04-30T06:13:38.814ZDEBGomicron-dev (store): query key key = _mgs._tcp
154002025-04-30T06:13:38.814ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
154012025-04-30T06:13:38.814ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:59195 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8ef382d5-f10f-4cc9-93f2-a6596b179a52
154022025-04-30T06:13:38.815ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2483, tv_nsec: 613933403 } })
154032025-04-30T06:13:38.815ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
154042025-04-30T06:13:38.815ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d3 03 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54615 req_id = 71cebbac-3780-4a17-a963-71141256b38b
154052025-04-30T06:13:38.815ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54019,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54615 req_id = 71cebbac-3780-4a17-a963-71141256b38b
154062025-04-30T06:13:38.816ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
154072025-04-30T06:13:38.816ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54615 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 71cebbac-3780-4a17-a963-71141256b38b
154082025-04-30T06:13:38.843ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
154092025-04-30T06:13:38.869ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
154102025-04-30T06:13:38.870ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: c1 a4 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61417 req_id = 9fcb397b-8f31-4f25-9671-d4ec96fcc81a
154112025-04-30T06:13:38.870ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49572,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61417 req_id = 9fcb397b-8f31-4f25-9671-d4ec96fcc81a
154122025-04-30T06:13:38.871ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
154132025-04-30T06:13:38.871ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 49572, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61417 req_id = 9fcb397b-8f31-4f25-9671-d4ec96fcc81a
154142025-04-30T06:13:38.871ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
154152025-04-30T06:13:38.871ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
154162025-04-30T06:13:38.871ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
154172025-04-30T06:13:38.877ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5523feb4-5cc4-43ea-ba37-244df737ef00", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154182025-04-30T06:13:38.877ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
154192025-04-30T06:13:38.879ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "be6f66af-139e-46f5-97de-8289a372719c", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154202025-04-30T06:13:38.880ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
154212025-04-30T06:13:38.882ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c5b8a470-a644-4219-83ef-024771c72e83", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154222025-04-30T06:13:38.883ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
154232025-04-30T06:13:38.885ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0aa4884c-0af7-49c7-bbc5-11136bff937a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154242025-04-30T06:13:38.885ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
154252025-04-30T06:13:38.887ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "47403b24-77a7-456e-b07e-149230ba2762", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154262025-04-30T06:13:38.888ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
154272025-04-30T06:13:38.891ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bd8a253a-5995-4b6a-aa59-94cf60256ddd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154282025-04-30T06:13:38.892ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
154292025-04-30T06:13:38.894ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
154302025-04-30T06:13:38.894ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
154312025-04-30T06:13:38.895ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
154322025-04-30T06:13:38.895ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
154332025-04-30T06:13:38.896ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
154342025-04-30T06:13:38.906ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d2cc850f-aff2-432c-bb5f-ae170b4fb85e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154352025-04-30T06:13:38.907ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
154362025-04-30T06:13:38.917ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "33a789e0-ede9-450a-a5b5-d02066aa9fc4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154372025-04-30T06:13:38.917ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
154382025-04-30T06:13:38.924ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c37dff5-82b9-4229-a8e3-39f53d851923", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154392025-04-30T06:13:38.925ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
154402025-04-30T06:13:38.935ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
154412025-04-30T06:13:38.941ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "135b3ef5-0877-405c-928f-5b1a70c7ce50", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154422025-04-30T06:13:38.942ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
154432025-04-30T06:13:38.944ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5ca7cc33-e79e-41ab-b3b9-cda6e2105316", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154442025-04-30T06:13:38.945ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
154452025-04-30T06:13:38.947ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "25477417-e0d4-4b0e-b43e-7437af994a8c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154462025-04-30T06:13:38.949ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
154472025-04-30T06:13:38.951ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "370c0f2e-33c9-4094-9535-4ceeefe8bbf0", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154482025-04-30T06:13:38.951ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
154492025-04-30T06:13:38.954ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fab468cf-2ac1-4ba0-b1d7-91af793d0e46", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154502025-04-30T06:13:38.954ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
154512025-04-30T06:13:38.956ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d526dd7b-8c69-4bb5-96bc-73bc1ea12415", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154522025-04-30T06:13:38.957ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
154532025-04-30T06:13:38.962ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0dd9071-9ffa-4626-91b0-345920a266d3", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154542025-04-30T06:13:38.962ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
154552025-04-30T06:13:38.970ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb2d9f5b-d272-4454-9bf2-e0fc46e52746", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154562025-04-30T06:13:38.970ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
154572025-04-30T06:13:38.972ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc84bd07-33ad-4005-ae7b-1313ea3484f6", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154582025-04-30T06:13:38.972ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
154592025-04-30T06:13:38.974ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
154602025-04-30T06:13:38.974ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
154612025-04-30T06:13:38.974ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
154622025-04-30T06:13:38.974ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
154632025-04-30T06:13:38.976ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a14bb40d-0b80-41d2-beaf-4cf727bfc735", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154642025-04-30T06:13:38.976ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
154652025-04-30T06:13:38.977ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a59bd43-c0b5-4139-bf69-ca9781f811db", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154662025-04-30T06:13:38.977ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
154672025-04-30T06:13:38.979ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fee08b07-9683-4fb2-999d-beef053379f3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154682025-04-30T06:13:38.979ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
154692025-04-30T06:13:38.986ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
154702025-04-30T06:13:38.986ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
154712025-04-30T06:13:38.988ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
154722025-04-30T06:13:38.996ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "85adb27b-6e3b-4183-8c78-fb5d21a313d4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154732025-04-30T06:13:38.997ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
154742025-04-30T06:13:39.001ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f87b07e7-f6c6-4f69-b4ce-dfeed6ab2e91", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154752025-04-30T06:13:39.004ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
154762025-04-30T06:13:39.007ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5fb9ad7d-8656-4ed1-b2be-a118a9573474", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154772025-04-30T06:13:39.007ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
154782025-04-30T06:13:39.009ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3587858c-0e45-41ea-a667-7edadf9ea59c", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154792025-04-30T06:13:39.011ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
154802025-04-30T06:13:39.013ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "db646960-9003-432d-bc68-3489e960f5a3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154812025-04-30T06:13:39.013ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
154822025-04-30T06:13:39.016ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d941c24e-2af7-4249-96d0-acdb1a06ede2", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154832025-04-30T06:13:39.016ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
154842025-04-30T06:13:39.018ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bccc4d2-60ff-431d-bd53-0986201fcbf6", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154852025-04-30T06:13:39.018ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
154862025-04-30T06:13:39.020ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "528fa8fa-1c04-4ef6-8b2a-89b19a12faa0", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154872025-04-30T06:13:39.021ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
154882025-04-30T06:13:39.023ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1c71918e-a4b9-4ee0-9205-3782bb73124a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154892025-04-30T06:13:39.023ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
154902025-04-30T06:13:39.024ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b0f5239-faff-4b8b-b44c-24dceedf9fa5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154912025-04-30T06:13:39.024ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
154922025-04-30T06:13:39.028ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "578f839f-e186-4200-8bd2-a52abc21c4a2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154932025-04-30T06:13:39.029ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
154942025-04-30T06:13:39.035ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c36f23a3-b066-4ffd-a0d7-67c326334b14", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154952025-04-30T06:13:39.035ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
154962025-04-30T06:13:39.045ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c405ec71-a4ec-45a3-85b6-b7fd5fc5449e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154972025-04-30T06:13:39.045ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
154982025-04-30T06:13:39.047ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3b13b1c8-a27f-42b5-a2a2-3fd352b39b84", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
154992025-04-30T06:13:39.047ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
155002025-04-30T06:13:39.050ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f47ee9e2-5697-45b2-9abe-bee8d8a18f02", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155012025-04-30T06:13:39.050ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
155022025-04-30T06:13:39.056ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1504f50d-329b-4e9b-9cde-2912dae59c40", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155032025-04-30T06:13:39.057ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
155042025-04-30T06:13:39.057ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a964ccd-7581-42ac-a780-7bc24c0b428d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155052025-04-30T06:13:39.057ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
155062025-04-30T06:13:39.058ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "23f55d68-45ae-47e1-a8db-3ddeae715125", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155072025-04-30T06:13:39.058ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
155082025-04-30T06:13:39.060ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "59b4ae65-467f-4525-9b4c-bae588a32ca9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155092025-04-30T06:13:39.060ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
155102025-04-30T06:13:39.062ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6104d416-b91e-4182-b60c-2e501828dff2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155112025-04-30T06:13:39.063ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
155122025-04-30T06:13:39.064ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca6b1b59-cb48-4ed2-b394-ab1ebf1a4c97", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155132025-04-30T06:13:39.064ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
155142025-04-30T06:13:39.065ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "78ad0256-d79e-4f8d-a23a-1c0f83625fcc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155152025-04-30T06:13:39.065ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
155162025-04-30T06:13:39.066ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3105d5d-80d1-4a32-8e9b-e992b92deae0", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155172025-04-30T06:13:39.066ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
155182025-04-30T06:13:39.067ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5cd7330a-a0e2-4044-b8b6-c992614019e5", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:38 GMT"} })
155192025-04-30T06:13:39.068ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
155202025-04-30T06:13:39.068ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
155212025-04-30T06:13:39.068ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
155222025-04-30T06:13:39.068ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
155232025-04-30T06:13:39.069ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
155242025-04-30T06:13:39.076ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155252025-04-30T06:13:39.085ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
155262025-04-30T06:13:39.087ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
155272025-04-30T06:13:39.087ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
155282025-04-30T06:13:39.087ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
155292025-04-30T06:13:39.087ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
155302025-04-30T06:13:39.087ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
155312025-04-30T06:13:39.088ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
155322025-04-30T06:13:39.096ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
155332025-04-30T06:13:39.097ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
155342025-04-30T06:13:39.097ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
155352025-04-30T06:13:39.099ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
155362025-04-30T06:13:39.101ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155372025-04-30T06:13:39.104ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
155382025-04-30T06:13:39.104ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
155392025-04-30T06:13:39.111ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
155402025-04-30T06:13:39.111ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
155412025-04-30T06:13:39.113ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
155422025-04-30T06:13:39.113ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
155432025-04-30T06:13:39.113ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
155442025-04-30T06:13:39.114ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
155452025-04-30T06:13:39.145ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
155462025-04-30T06:13:39.145ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
155472025-04-30T06:13:39.145ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155482025-04-30T06:13:39.145ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
155492025-04-30T06:13:39.146ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155502025-04-30T06:13:39.199ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
155512025-04-30T06:13:39.209ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
155522025-04-30T06:13:39.210ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
155532025-04-30T06:13:39.210ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:58969 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
155542025-04-30T06:13:39.213ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155552025-04-30T06:13:39.216ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:58969 req_id = 8cf88594-8b36-4cf6-b669-2cf8db0b1e29 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
155562025-04-30T06:13:39.218ZINFOomicron-dev (dropshot): request completed latency_us = 1663 local_addr = [::1]:61881 method = GET remote_addr = [::1]:58969 req_id = 8cf88594-8b36-4cf6-b669-2cf8db0b1e29 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
155572025-04-30T06:13:39.218ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cf88594-8b36-4cf6-b669-2cf8db0b1e29", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
155582025-04-30T06:13:39.221ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
155592025-04-30T06:13:39.221ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155602025-04-30T06:13:39.221ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
155612025-04-30T06:13:39.222ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155622025-04-30T06:13:39.222ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
155632025-04-30T06:13:39.222ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
155642025-04-30T06:13:39.222ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
155652025-04-30T06:13:39.225ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
155662025-04-30T06:13:39.225ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
155672025-04-30T06:13:39.225ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
155682025-04-30T06:13:39.225ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
155692025-04-30T06:13:39.226ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155702025-04-30T06:13:39.226ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
155712025-04-30T06:13:39.226ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
155722025-04-30T06:13:39.228ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155732025-04-30T06:13:39.232ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155742025-04-30T06:13:39.239ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
155752025-04-30T06:13:39.239ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
155762025-04-30T06:13:39.239ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
155772025-04-30T06:13:39.240ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
155782025-04-30T06:13:39.257ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
155792025-04-30T06:13:39.285ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:63163 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
155802025-04-30T06:13:39.286ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:63163 req_id = fab00988-abde-4fba-80f0-85dc104fd5b7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
155812025-04-30T06:13:39.286ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 844 local_addr = [::1]:55020 method = GET remote_addr = [::1]:63163 req_id = fab00988-abde-4fba-80f0-85dc104fd5b7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
155822025-04-30T06:13:39.307ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
155832025-04-30T06:13:39.307ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
155842025-04-30T06:13:39.307ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 513.454846ms iteration = 58 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
155852025-04-30T06:13:39.307ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 59 reason = Signaled
155862025-04-30T06:13:39.307ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
155872025-04-30T06:13:39.307ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
155882025-04-30T06:13:39.307ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
155892025-04-30T06:13:39.308ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
155902025-04-30T06:13:39.308ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
155912025-04-30T06:13:39.309ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
155922025-04-30T06:13:39.309ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
155932025-04-30T06:13:39.309ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
155942025-04-30T06:13:39.309ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
155952025-04-30T06:13:39.310ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
155962025-04-30T06:13:39.312ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155972025-04-30T06:13:39.314ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
155982025-04-30T06:13:39.319ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
155992025-04-30T06:13:39.319ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
156002025-04-30T06:13:39.319ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
156012025-04-30T06:13:39.320ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
156022025-04-30T06:13:39.321ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
156032025-04-30T06:13:39.321ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156042025-04-30T06:13:39.321ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
156052025-04-30T06:13:39.321ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156062025-04-30T06:13:39.323ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
156072025-04-30T06:13:39.323ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
156082025-04-30T06:13:39.324ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 71 8b 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 q............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:56576 req_id = 8788a9c6-961d-4bba-a15e-e68e75e32666
156092025-04-30T06:13:39.324ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29067,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56576 req_id = 8788a9c6-961d-4bba-a15e-e68e75e32666
156102025-04-30T06:13:39.324ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156112025-04-30T06:13:39.324ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
156122025-04-30T06:13:39.324ZDEBGomicron-dev (store): query key key = _mgs._tcp
156132025-04-30T06:13:39.324ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156142025-04-30T06:13:39.324ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:56576 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8788a9c6-961d-4bba-a15e-e68e75e32666
156152025-04-30T06:13:39.325ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156162025-04-30T06:13:39.326ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2484, tv_nsec: 124251057 } })
156172025-04-30T06:13:39.326ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
156182025-04-30T06:13:39.327ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ad 7e 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .~..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:65042 req_id = 3f824b3d-199d-4376-ae82-a7f06657180a
156192025-04-30T06:13:39.327ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44414,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65042 req_id = 3f824b3d-199d-4376-ae82-a7f06657180a
156202025-04-30T06:13:39.327ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156212025-04-30T06:13:39.327ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:65042 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3f824b3d-199d-4376-ae82-a7f06657180a
156222025-04-30T06:13:39.366ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
156232025-04-30T06:13:39.370ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
156242025-04-30T06:13:39.371ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: d8 29 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .)..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:64355 req_id = de9e842c-167b-4517-b7b3-ce88711a782d
156252025-04-30T06:13:39.371ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 55337,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64355 req_id = de9e842c-167b-4517-b7b3-ce88711a782d
156262025-04-30T06:13:39.371ZDEBGomicron-dev (store): query key key = _dendrite._tcp
156272025-04-30T06:13:39.371ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156282025-04-30T06:13:39.371ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64355 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = de9e842c-167b-4517-b7b3-ce88711a782d
156292025-04-30T06:13:39.372ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2484, tv_nsec: 171215712 } })
156302025-04-30T06:13:39.372ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
156312025-04-30T06:13:39.372ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: cb 6b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .k..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57285 req_id = 15c77527-7e0d-4bbe-a8fe-6516abe79825
156322025-04-30T06:13:39.372ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 52075,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57285 req_id = 15c77527-7e0d-4bbe-a8fe-6516abe79825
156332025-04-30T06:13:39.372ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156342025-04-30T06:13:39.372ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57285 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 15c77527-7e0d-4bbe-a8fe-6516abe79825
156352025-04-30T06:13:39.373ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
156362025-04-30T06:13:39.384ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
156372025-04-30T06:13:39.386ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: dd 3e 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .>..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:52395 req_id = 1dfa2620-cc7e-437f-a576-3ac1b4946b82
156382025-04-30T06:13:39.386ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 56638,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52395 req_id = 1dfa2620-cc7e-437f-a576-3ac1b4946b82
156392025-04-30T06:13:39.386ZDEBGomicron-dev (store): query key key = _dendrite._tcp
156402025-04-30T06:13:39.386ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156412025-04-30T06:13:39.386ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52395 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1dfa2620-cc7e-437f-a576-3ac1b4946b82
156422025-04-30T06:13:39.387ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2484, tv_nsec: 186723770 } })
156432025-04-30T06:13:39.388ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
156442025-04-30T06:13:39.389ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: d5 4c 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .L..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:52621 req_id = 1eebb351-6e1e-4db8-8be8-dc0a261a3e24
156452025-04-30T06:13:39.389ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54604,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52621 req_id = 1eebb351-6e1e-4db8-8be8-dc0a261a3e24
156462025-04-30T06:13:39.390ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156472025-04-30T06:13:39.390ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:52621 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1eebb351-6e1e-4db8-8be8-dc0a261a3e24
156482025-04-30T06:13:39.390ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
156492025-04-30T06:13:39.418ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
156502025-04-30T06:13:39.418ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
156512025-04-30T06:13:39.418ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156522025-04-30T06:13:39.418ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
156532025-04-30T06:13:39.420ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156542025-04-30T06:13:39.425ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156552025-04-30T06:13:39.434ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
156562025-04-30T06:13:39.434ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156572025-04-30T06:13:39.434ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
156582025-04-30T06:13:39.435ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156592025-04-30T06:13:39.444ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156602025-04-30T06:13:39.445ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
156612025-04-30T06:13:39.445ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156622025-04-30T06:13:39.485ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
156632025-04-30T06:13:39.492ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
156642025-04-30T06:13:39.493ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 27 5a 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 'Z..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:64585 req_id = 8b126679-ab5d-4370-aa01-fb9629110752
156652025-04-30T06:13:39.493ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10074,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64585 req_id = 8b126679-ab5d-4370-aa01-fb9629110752
156662025-04-30T06:13:39.493ZDEBGomicron-dev (store): query key key = _dendrite._tcp
156672025-04-30T06:13:39.493ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156682025-04-30T06:13:39.493ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64585 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 8b126679-ab5d-4370-aa01-fb9629110752
156692025-04-30T06:13:39.494ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2484, tv_nsec: 293112242 } })
156702025-04-30T06:13:39.494ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
156712025-04-30T06:13:39.494ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 5c e3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e \\...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63478 req_id = 0fd73237-199c-4ec0-863c-c0572d32a652
156722025-04-30T06:13:39.494ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23779,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63478 req_id = 0fd73237-199c-4ec0-863c-c0572d32a652
156732025-04-30T06:13:39.494ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
156742025-04-30T06:13:39.494ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63478 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0fd73237-199c-4ec0-863c-c0572d32a652
156752025-04-30T06:13:39.495ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
156762025-04-30T06:13:39.547ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
156772025-04-30T06:13:39.547ZDEBGomicron-dev (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
156782025-04-30T06:13:39.548ZDEBGomicron-dev (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
156792025-04-30T06:13:39.548ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
156802025-04-30T06:13:39.549ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
156812025-04-30T06:13:39.549ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
156822025-04-30T06:13:39.549ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156832025-04-30T06:13:39.549ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
156842025-04-30T06:13:39.550ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156852025-04-30T06:13:39.554ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
156862025-04-30T06:13:39.559ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
156872025-04-30T06:13:39.559ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156882025-04-30T06:13:39.559ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
156892025-04-30T06:13:39.560ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156902025-04-30T06:13:39.564ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
156912025-04-30T06:13:39.564ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
156922025-04-30T06:13:39.565ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
156932025-04-30T06:13:39.579ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
156942025-04-30T06:13:39.581ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: e0 86 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61610 req_id = bef3c893-f251-4b12-b471-df1cfc4a689f
156952025-04-30T06:13:39.581ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 57478,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61610 req_id = bef3c893-f251-4b12-b471-df1cfc4a689f
156962025-04-30T06:13:39.581ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
156972025-04-30T06:13:39.581ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 57478, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61610 req_id = bef3c893-f251-4b12-b471-df1cfc4a689f
156982025-04-30T06:13:39.582ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
156992025-04-30T06:13:39.582ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
157002025-04-30T06:13:39.582ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
157012025-04-30T06:13:39.586ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2e010ee-84ed-429f-8985-f2e7bfc4aea8", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157022025-04-30T06:13:39.587ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
157032025-04-30T06:13:39.589ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "eaa8eaca-525b-4d1b-9657-c481b44dc7e2", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157042025-04-30T06:13:39.589ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
157052025-04-30T06:13:39.589ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
157062025-04-30T06:13:39.589ZDEBGomicron-dev (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
157072025-04-30T06:13:39.597ZDEBGomicron-dev (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
157082025-04-30T06:13:39.597ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
157092025-04-30T06:13:39.597ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
157102025-04-30T06:13:39.608ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8c97f686-5d16-40b1-8537-9bba4c4db28d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157112025-04-30T06:13:39.608ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
157122025-04-30T06:13:39.612ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
157132025-04-30T06:13:39.612ZDEBGomicron-dev (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
157142025-04-30T06:13:39.612ZDEBGomicron-dev (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
157152025-04-30T06:13:39.612ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
157162025-04-30T06:13:39.612ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "760d6220-24dc-491c-86f3-8fb48c89eeae", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157172025-04-30T06:13:39.612ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
157182025-04-30T06:13:39.613ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
157192025-04-30T06:13:39.614ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
157202025-04-30T06:13:39.615ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
157212025-04-30T06:13:39.617ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2f6197b-b4e0-429f-a4d8-95c6468be6d8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157222025-04-30T06:13:39.617ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
157232025-04-30T06:13:39.623ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7978d439-9292-4f28-9816-f76164e43621", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157242025-04-30T06:13:39.623ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
157252025-04-30T06:13:39.628ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "40ba6eb5-a071-4068-ba80-64877ebcb699", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157262025-04-30T06:13:39.628ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
157272025-04-30T06:13:39.636ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3a292b47-94da-4a81-97d0-388121eea0c7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157282025-04-30T06:13:39.636ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
157292025-04-30T06:13:39.637ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2fa82ae9-5206-4f1c-bfa9-b7a8e60f0fb0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157302025-04-30T06:13:39.637ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
157312025-04-30T06:13:39.638ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5eef5e53-7c80-4e6e-9897-c371d5311cec", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157322025-04-30T06:13:39.639ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
157332025-04-30T06:13:39.640ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5dde0105-8b1d-4fb0-b036-69a9835cdbce", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157342025-04-30T06:13:39.640ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
157352025-04-30T06:13:39.641ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9fb59b5e-1da7-4093-b21f-eb3f467aba79", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157362025-04-30T06:13:39.641ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
157372025-04-30T06:13:39.642ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "202334ca-754e-4ba6-9216-e9ff60910c54", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157382025-04-30T06:13:39.642ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
157392025-04-30T06:13:39.644ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "91a04793-62d4-4da0-89aa-a6fbb92fc250", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157402025-04-30T06:13:39.644ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
157412025-04-30T06:13:39.646ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "613cccff-0c5e-4945-b621-5312a7e09ab7", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157422025-04-30T06:13:39.646ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
157432025-04-30T06:13:39.648ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "abc9c379-e830-42cb-8faf-5c31e7cf4748", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157442025-04-30T06:13:39.648ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
157452025-04-30T06:13:39.656ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0769263c-63e4-4105-8dfc-70572985e666", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157462025-04-30T06:13:39.656ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
157472025-04-30T06:13:39.659ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "53dd921c-2df3-46d1-a412-5498ee60a52d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157482025-04-30T06:13:39.659ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
157492025-04-30T06:13:39.663ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "40628868-b344-4ddc-82bc-6ffc215ae25b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157502025-04-30T06:13:39.663ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
157512025-04-30T06:13:39.664ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3225f1c-3cc2-4265-b8f5-3515218d7fb6", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157522025-04-30T06:13:39.666ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
157532025-04-30T06:13:39.666ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
157542025-04-30T06:13:39.666ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
157552025-04-30T06:13:39.666ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
157562025-04-30T06:13:39.666ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
157572025-04-30T06:13:39.667ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
157582025-04-30T06:13:39.670ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b457206-6e7c-4a98-89c9-8b6662a28b0a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157592025-04-30T06:13:39.670ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
157602025-04-30T06:13:39.672ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
157612025-04-30T06:13:39.677ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
157622025-04-30T06:13:39.677ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
157632025-04-30T06:13:39.677ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
157642025-04-30T06:13:39.677ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
157652025-04-30T06:13:39.680ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ef1d273e-6a43-4c48-acd6-cd384676e2a4", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157662025-04-30T06:13:39.680ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
157672025-04-30T06:13:39.681ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
157682025-04-30T06:13:39.681ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
157692025-04-30T06:13:39.681ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
157702025-04-30T06:13:39.697ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5312701-fdf4-402f-9570-558464e84ce1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157712025-04-30T06:13:39.700ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
157722025-04-30T06:13:39.711ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f5983360-4c1e-468c-9a2b-ac969b17a018", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157732025-04-30T06:13:39.711ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
157742025-04-30T06:13:39.716ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "372593e6-b4e4-45cb-a960-d19047f576df", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157752025-04-30T06:13:39.716ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
157762025-04-30T06:13:39.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
157772025-04-30T06:13:39.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
157782025-04-30T06:13:39.722ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "463678df-ac9a-4520-a9db-9f4edece1b97", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157792025-04-30T06:13:39.722ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
157802025-04-30T06:13:39.723ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
157812025-04-30T06:13:39.724ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
157822025-04-30T06:13:39.724ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
157832025-04-30T06:13:39.724ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
157842025-04-30T06:13:39.727ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
157852025-04-30T06:13:39.728ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "309d96a1-eab6-459a-aceb-6a0696c57048", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157862025-04-30T06:13:39.728ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
157872025-04-30T06:13:39.732ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92fb4cff-8c8e-48aa-b1a5-7ce1c2717238", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157882025-04-30T06:13:39.733ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
157892025-04-30T06:13:39.736ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a7b0674-0993-4b90-8a40-b7e9f055be6e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157902025-04-30T06:13:39.737ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
157912025-04-30T06:13:39.742ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b92b2940-3489-43c4-a74e-2c9170640f2f", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157922025-04-30T06:13:39.742ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
157932025-04-30T06:13:39.743ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8194d3b9-dd1d-490e-9156-e8366c008bf3", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157942025-04-30T06:13:39.743ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
157952025-04-30T06:13:39.745ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a42d1a2-f970-4876-a679-25eec1ef2154", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157962025-04-30T06:13:39.745ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
157972025-04-30T06:13:39.747ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8d268796-d993-4661-a988-158067321ada", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
157982025-04-30T06:13:39.747ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
157992025-04-30T06:13:39.748ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1148cc6-e516-4675-bbe3-04fe49aac133", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158002025-04-30T06:13:39.748ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
158012025-04-30T06:13:39.750ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb667d17-7fac-4045-8f58-a66db0fb24e5", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158022025-04-30T06:13:39.750ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
158032025-04-30T06:13:39.754ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25bf8e48-9d2a-4e18-a96c-59bd05e6af71", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158042025-04-30T06:13:39.754ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
158052025-04-30T06:13:39.757ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3bf8b8a0-4df1-430d-8347-7469e2cabbdc", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158062025-04-30T06:13:39.757ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
158072025-04-30T06:13:39.759ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d997f563-ebc5-4ffb-a7b1-27eddd98a0aa", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158082025-04-30T06:13:39.759ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
158092025-04-30T06:13:39.779ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
158102025-04-30T06:13:39.779ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158112025-04-30T06:13:39.779ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
158122025-04-30T06:13:39.779ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
158132025-04-30T06:13:39.780ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
158142025-04-30T06:13:39.797ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158152025-04-30T06:13:39.798ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f920c87b-0523-4964-ba99-c2f252cb4f06", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158162025-04-30T06:13:39.798ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
158172025-04-30T06:13:39.801ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b875c40a-f3b4-478c-ad87-77a4a87f7b9c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158182025-04-30T06:13:39.801ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
158192025-04-30T06:13:39.806ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d4ad18f-1403-403e-a7f0-7b1d73d7bfdb", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158202025-04-30T06:13:39.806ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
158212025-04-30T06:13:39.810ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "60767109-92a4-446c-929b-ad72d8cd61ce", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158222025-04-30T06:13:39.811ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
158232025-04-30T06:13:39.817ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
158242025-04-30T06:13:39.817ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
158252025-04-30T06:13:39.817ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
158262025-04-30T06:13:39.818ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
158272025-04-30T06:13:39.818ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6ad3b7aa-f521-47a0-bb42-c79a73ab9dbc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158282025-04-30T06:13:39.818ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
158292025-04-30T06:13:39.819ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bf29faf-65cb-4c7c-a17e-a449120f85a7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158302025-04-30T06:13:39.820ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
158312025-04-30T06:13:39.821ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2975828-f6a6-49d3-8a4b-fbc2028c0de0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158322025-04-30T06:13:39.821ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
158332025-04-30T06:13:39.821ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158342025-04-30T06:13:39.821ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158352025-04-30T06:13:39.821ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158362025-04-30T06:13:39.821ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
158372025-04-30T06:13:39.822ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
158382025-04-30T06:13:39.822ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158392025-04-30T06:13:39.823ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
158402025-04-30T06:13:39.841ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158412025-04-30T06:13:39.845ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
158422025-04-30T06:13:39.845ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
158432025-04-30T06:13:39.845ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158442025-04-30T06:13:39.845ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158452025-04-30T06:13:39.845ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158462025-04-30T06:13:39.846ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158472025-04-30T06:13:39.861ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158482025-04-30T06:13:39.861ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:57328 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
158492025-04-30T06:13:39.862ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:57328 req_id = df32bb2d-51a1-4e80-91dc-207949e27caf sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
158502025-04-30T06:13:39.862ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 234 local_addr = [::1]:55020 method = GET remote_addr = [::1]:57328 req_id = df32bb2d-51a1-4e80-91dc-207949e27caf response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
158512025-04-30T06:13:39.864ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
158522025-04-30T06:13:39.864ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158532025-04-30T06:13:39.864ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158542025-04-30T06:13:39.865ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158552025-04-30T06:13:39.933ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
158562025-04-30T06:13:39.933ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
158572025-04-30T06:13:39.934ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:61991 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
158582025-04-30T06:13:39.935ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:61991 req_id = 388c67a5-cec2-4c5d-9666-ae5a8ca19b77 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
158592025-04-30T06:13:39.935ZINFOomicron-dev (dropshot): request completed latency_us = 548 local_addr = [::1]:61881 method = GET remote_addr = [::1]:61991 req_id = 388c67a5-cec2-4c5d-9666-ae5a8ca19b77 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
158602025-04-30T06:13:39.936ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "388c67a5-cec2-4c5d-9666-ae5a8ca19b77", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
158612025-04-30T06:13:39.939ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
158622025-04-30T06:13:39.940ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
158632025-04-30T06:13:39.940ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
158642025-04-30T06:13:39.940ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
158652025-04-30T06:13:39.945ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
158662025-04-30T06:13:39.945ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158672025-04-30T06:13:39.945ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158682025-04-30T06:13:39.945ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158692025-04-30T06:13:39.946ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158702025-04-30T06:13:39.953ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158712025-04-30T06:13:39.959ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
158722025-04-30T06:13:39.959ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158732025-04-30T06:13:39.959ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158742025-04-30T06:13:39.960ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158752025-04-30T06:13:39.994ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
158762025-04-30T06:13:39.994ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158772025-04-30T06:13:39.994ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
158782025-04-30T06:13:39.994ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
158792025-04-30T06:13:39.995ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
158802025-04-30T06:13:40.006ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158812025-04-30T06:13:40.011ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
158822025-04-30T06:13:40.012ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
158832025-04-30T06:13:40.012ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 704.579398ms iteration = 59 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
158842025-04-30T06:13:40.012ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 60 reason = Signaled
158852025-04-30T06:13:40.012ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
158862025-04-30T06:13:40.012ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
158872025-04-30T06:13:40.012ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158882025-04-30T06:13:40.012ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158892025-04-30T06:13:40.014ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158902025-04-30T06:13:40.016ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
158912025-04-30T06:13:40.016ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
158922025-04-30T06:13:40.016ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
158932025-04-30T06:13:40.017ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
158942025-04-30T06:13:40.018ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
158952025-04-30T06:13:40.026ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
158962025-04-30T06:13:40.027ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
158972025-04-30T06:13:40.027ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
158982025-04-30T06:13:40.027ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
158992025-04-30T06:13:40.033ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
159002025-04-30T06:13:40.033ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
159012025-04-30T06:13:40.033ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
159022025-04-30T06:13:40.034ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
159032025-04-30T06:13:40.034ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: a5 e0 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:60273 req_id = 5d51a500-24fc-4acb-9ac6-12d669a044c9
159042025-04-30T06:13:40.034ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 42464,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60273 req_id = 5d51a500-24fc-4acb-9ac6-12d669a044c9
159052025-04-30T06:13:40.034ZDEBGomicron-dev (store): query key key = _mgs._tcp
159062025-04-30T06:13:40.036ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
159072025-04-30T06:13:40.036ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:60273 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5d51a500-24fc-4acb-9ac6-12d669a044c9
159082025-04-30T06:13:40.036ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2484, tv_nsec: 835580475 } })
159092025-04-30T06:13:40.036ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
159102025-04-30T06:13:40.037ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ea a3 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:53648 req_id = f5ddc3f5-5017-4999-a445-a7246de504a9
159112025-04-30T06:13:40.037ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 60067,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53648 req_id = f5ddc3f5-5017-4999-a445-a7246de504a9
159122025-04-30T06:13:40.037ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
159132025-04-30T06:13:40.037ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:53648 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = f5ddc3f5-5017-4999-a445-a7246de504a9
159142025-04-30T06:13:40.040ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
159152025-04-30T06:13:40.090ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
159162025-04-30T06:13:40.090ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
159172025-04-30T06:13:40.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
159182025-04-30T06:13:40.091ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
159192025-04-30T06:13:40.099ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
159202025-04-30T06:13:40.101ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: c5 75 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .u..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:54093 req_id = 18b9deb7-c34b-4486-b4bd-65f7d134a023
159212025-04-30T06:13:40.101ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50549,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54093 req_id = 18b9deb7-c34b-4486-b4bd-65f7d134a023
159222025-04-30T06:13:40.101ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
159232025-04-30T06:13:40.101ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
159242025-04-30T06:13:40.101ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
159252025-04-30T06:13:40.101ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 50549, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:54093 req_id = 18b9deb7-c34b-4486-b4bd-65f7d134a023
159262025-04-30T06:13:40.102ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
159272025-04-30T06:13:40.102ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
159282025-04-30T06:13:40.102ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
159292025-04-30T06:13:40.108ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
159302025-04-30T06:13:40.108ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
159312025-04-30T06:13:40.109ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "61230e94-2979-4e21-abcc-6507549b7705", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159322025-04-30T06:13:40.109ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
159332025-04-30T06:13:40.112ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1351e955-c585-4fa5-a162-2ca12b825669", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159342025-04-30T06:13:40.113ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
159352025-04-30T06:13:40.116ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "222a5991-7eef-4fbb-8d9b-e598151b9bd3", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159362025-04-30T06:13:40.116ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
159372025-04-30T06:13:40.118ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6363eee9-9d2d-4737-8f73-1ed009d4304d", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159382025-04-30T06:13:40.118ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
159392025-04-30T06:13:40.120ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "02288dbe-3d95-4f76-9c30-fff4455cff7f", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159402025-04-30T06:13:40.121ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
159412025-04-30T06:13:40.121ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
159422025-04-30T06:13:40.124ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "54795aea-7799-4c28-830e-420447c9549a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159432025-04-30T06:13:40.124ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
159442025-04-30T06:13:40.128ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a961e91c-0c39-485c-a007-b8f120108ad0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159452025-04-30T06:13:40.128ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
159462025-04-30T06:13:40.130ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e5fa0ba3-f640-4bfc-a7ee-39a9e2bdc4bf", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159472025-04-30T06:13:40.131ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
159482025-04-30T06:13:40.131ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa927ed8-6a82-4ddf-af92-dae3f9f5d3d8", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159492025-04-30T06:13:40.132ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
159502025-04-30T06:13:40.133ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d3e201a-0528-4760-a415-0076076e5252", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159512025-04-30T06:13:40.134ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
159522025-04-30T06:13:40.136ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "23bcceaa-2d39-4855-b998-f581186c4744", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159532025-04-30T06:13:40.136ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
159542025-04-30T06:13:40.138ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ed4c2d3-434f-4706-8efb-c550717d4af1", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159552025-04-30T06:13:40.139ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
159562025-04-30T06:13:40.140ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f88d70d2-8633-4022-a411-71a2ad6def03", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159572025-04-30T06:13:40.140ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
159582025-04-30T06:13:40.145ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dfe68fd3-56fb-4c94-8080-b7acb844ffa8", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159592025-04-30T06:13:40.145ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
159602025-04-30T06:13:40.147ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "22b715b7-9919-4d03-899d-6bf54eee4bf9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159612025-04-30T06:13:40.147ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
159622025-04-30T06:13:40.150ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f43d5149-f3fd-488b-ac1a-f70fdd738942", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159632025-04-30T06:13:40.150ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
159642025-04-30T06:13:40.153ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3a17132-940a-4635-a3cd-1c1282b8d699", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159652025-04-30T06:13:40.153ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
159662025-04-30T06:13:40.155ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "aa539368-013f-4d18-8572-ed99573da82e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159672025-04-30T06:13:40.155ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
159682025-04-30T06:13:40.158ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "af5ccee3-4c5f-48a0-aa51-9a13b57f3b2c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159692025-04-30T06:13:40.158ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
159702025-04-30T06:13:40.169ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b0b89ed2-37f7-48c7-a8a6-537e7dfb2843", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159712025-04-30T06:13:40.169ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
159722025-04-30T06:13:40.172ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
159732025-04-30T06:13:40.172ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
159742025-04-30T06:13:40.172ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
159752025-04-30T06:13:40.172ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
159762025-04-30T06:13:40.172ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8ea54210-6ce1-41cb-826c-eaadbb91921a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159772025-04-30T06:13:40.172ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
159782025-04-30T06:13:40.173ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
159792025-04-30T06:13:40.175ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4141a687-ebce-4b82-a129-ec447be05cce", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159802025-04-30T06:13:40.175ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
159812025-04-30T06:13:40.181ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
159822025-04-30T06:13:40.181ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "29bc9abf-14c1-42a4-9f19-549b4d96c162", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159832025-04-30T06:13:40.181ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
159842025-04-30T06:13:40.181ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c95a022b-7d6a-4ffc-8680-2a6e7bc248db", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159852025-04-30T06:13:40.181ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
159862025-04-30T06:13:40.181ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1ef892ce-66a1-4f8e-873c-4782e7f970ab", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159872025-04-30T06:13:40.181ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
159882025-04-30T06:13:40.183ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
159892025-04-30T06:13:40.183ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
159902025-04-30T06:13:40.183ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
159912025-04-30T06:13:40.183ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
159922025-04-30T06:13:40.185ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cee5e2f1-3ab9-4e22-a5b5-5789c9981024", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159932025-04-30T06:13:40.185ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
159942025-04-30T06:13:40.186ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
159952025-04-30T06:13:40.186ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
159962025-04-30T06:13:40.187ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
159972025-04-30T06:13:40.189ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "319d16fb-8187-4d98-ad39-30cb5a955894", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
159982025-04-30T06:13:40.189ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
159992025-04-30T06:13:40.191ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
160002025-04-30T06:13:40.191ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a166f28-1099-4685-b6d7-9d4bd0b1c26f", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160012025-04-30T06:13:40.191ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
160022025-04-30T06:13:40.194ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:52110 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
160032025-04-30T06:13:40.194ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:52110 req_id = 089a43e1-6c76-4553-851a-ecfcc6f11a5c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160042025-04-30T06:13:40.194ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 240 local_addr = [::1]:55020 method = GET remote_addr = [::1]:52110 req_id = 089a43e1-6c76-4553-851a-ecfcc6f11a5c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160052025-04-30T06:13:40.195ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "37c2f392-79f1-4e2e-8242-a84fd6b8f716", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160062025-04-30T06:13:40.196ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
160072025-04-30T06:13:40.197ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4b232cbb-b9d9-4fcb-9b77-5327b8f3e96a", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160082025-04-30T06:13:40.198ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
160092025-04-30T06:13:40.199ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9749b925-c258-41fa-9293-e46368a39965", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160102025-04-30T06:13:40.199ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
160112025-04-30T06:13:40.202ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1779f66-9b26-45a9-92dd-f997d760c490", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160122025-04-30T06:13:40.202ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
160132025-04-30T06:13:40.204ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "533db46b-c1e6-4077-86eb-8b54ec0df9da", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160142025-04-30T06:13:40.204ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
160152025-04-30T06:13:40.205ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f2e51d4f-2414-41a5-9117-9c24b42df16e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160162025-04-30T06:13:40.205ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
160172025-04-30T06:13:40.206ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a5f6cc1-fef1-4fbf-8258-9873365dea29", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160182025-04-30T06:13:40.206ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
160192025-04-30T06:13:40.208ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1bfad4ad-4ecb-4ff5-99c3-ccee725eec0f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160202025-04-30T06:13:40.208ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
160212025-04-30T06:13:40.210ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0131d9a-e030-4b5c-b176-e89daddcc7ac", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160222025-04-30T06:13:40.210ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
160232025-04-30T06:13:40.212ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "25545aef-c4ab-4bc6-9a74-8a6ce7a5014e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160242025-04-30T06:13:40.212ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
160252025-04-30T06:13:40.214ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bab69147-fd06-4029-be2b-9d2265641a3e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160262025-04-30T06:13:40.214ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
160272025-04-30T06:13:40.214ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
160282025-04-30T06:13:40.217ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e741ac89-21ca-48b2-9005-a485bcfbbded", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160292025-04-30T06:13:40.217ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
160302025-04-30T06:13:40.220ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "31080842-f03d-4c04-b482-c41abec74833", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160312025-04-30T06:13:40.220ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
160322025-04-30T06:13:40.223ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd2fbbce-84ca-463d-88d9-4c876192de3b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160332025-04-30T06:13:40.223ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
160342025-04-30T06:13:40.225ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "772bdaa3-667d-41ae-a236-cf7a0cee6944", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160352025-04-30T06:13:40.225ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
160362025-04-30T06:13:40.226ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cbdac7c2-5726-4807-86b2-915e9a53842a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160372025-04-30T06:13:40.226ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
160382025-04-30T06:13:40.229ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "383f4e6b-565f-4086-a807-d063c409f164", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160392025-04-30T06:13:40.229ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
160402025-04-30T06:13:40.229ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
160412025-04-30T06:13:40.229ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
160422025-04-30T06:13:40.229ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
160432025-04-30T06:13:40.230ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
160442025-04-30T06:13:40.234ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160452025-04-30T06:13:40.238ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
160462025-04-30T06:13:40.238ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
160472025-04-30T06:13:40.239ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
160482025-04-30T06:13:40.239ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
160492025-04-30T06:13:40.239ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
160502025-04-30T06:13:40.239ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
160512025-04-30T06:13:40.242ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160522025-04-30T06:13:40.245ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:41978 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
160532025-04-30T06:13:40.247ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:41978 req_id = 4c86063f-d629-4ecd-b473-f365757b5aff sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160542025-04-30T06:13:40.247ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 478 local_addr = [::1]:55020 method = GET remote_addr = [::1]:41978 req_id = 4c86063f-d629-4ecd-b473-f365757b5aff response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160552025-04-30T06:13:40.249ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:36222 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
160562025-04-30T06:13:40.250ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
160572025-04-30T06:13:40.250ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
160582025-04-30T06:13:40.250ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
160592025-04-30T06:13:40.250ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:36222 req_id = 764c0ee5-d132-4d85-823e-3f33de39ea64 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160602025-04-30T06:13:40.251ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 457 local_addr = [::1]:55020 method = GET remote_addr = [::1]:36222 req_id = 764c0ee5-d132-4d85-823e-3f33de39ea64 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160612025-04-30T06:13:40.251ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
160622025-04-30T06:13:40.265ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
160632025-04-30T06:13:40.265ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
160642025-04-30T06:13:40.265ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
160652025-04-30T06:13:40.266ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
160662025-04-30T06:13:40.267ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
160672025-04-30T06:13:40.292ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:53784 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
160682025-04-30T06:13:40.292ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:53784 req_id = baf2caee-7ffd-46a9-a209-a93c4fc56f64 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160692025-04-30T06:13:40.292ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 212 local_addr = [::1]:55020 method = GET remote_addr = [::1]:53784 req_id = baf2caee-7ffd-46a9-a209-a93c4fc56f64 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160702025-04-30T06:13:40.318ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
160712025-04-30T06:13:40.318ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
160722025-04-30T06:13:40.318ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:43939 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
160732025-04-30T06:13:40.319ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:43939 req_id = 5d3b0aa8-15a7-4b46-8630-b772b4ab3b8c sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
160742025-04-30T06:13:40.320ZINFOomicron-dev (dropshot): request completed latency_us = 704 local_addr = [::1]:61881 method = GET remote_addr = [::1]:43939 req_id = 5d3b0aa8-15a7-4b46-8630-b772b4ab3b8c response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
160752025-04-30T06:13:40.320ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "5d3b0aa8-15a7-4b46-8630-b772b4ab3b8c", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
160762025-04-30T06:13:40.322ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160772025-04-30T06:13:40.323ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
160782025-04-30T06:13:40.323ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
160792025-04-30T06:13:40.323ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
160802025-04-30T06:13:40.325ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
160812025-04-30T06:13:40.325ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
160822025-04-30T06:13:40.325ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
160832025-04-30T06:13:40.325ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
160842025-04-30T06:13:40.326ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
160852025-04-30T06:13:40.328ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
160862025-04-30T06:13:40.328ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
160872025-04-30T06:13:40.328ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
160882025-04-30T06:13:40.328ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
160892025-04-30T06:13:40.329ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
160902025-04-30T06:13:40.331ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
160912025-04-30T06:13:40.331ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
160922025-04-30T06:13:40.331ZINFOomicron-dev (dropshot): accepted connection kind = pantry local_addr = [::1]:55020 remote_addr = [::1]:64388 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
160932025-04-30T06:13:40.331ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
160942025-04-30T06:13:40.331ZTRAComicron-dev (dropshot): incoming request kind = pantry local_addr = [::1]:55020 method = GET remote_addr = [::1]:64388 req_id = bb800e00-f6be-4125-b6c4-3038995e4330 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160952025-04-30T06:13:40.332ZINFOomicron-dev (dropshot): request completed kind = pantry latency_us = 260 local_addr = [::1]:55020 method = GET remote_addr = [::1]:64388 req_id = bb800e00-f6be-4125-b6c4-3038995e4330 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /crucible/pantry/0
160962025-04-30T06:13:40.333ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
160972025-04-30T06:13:40.333ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
160982025-04-30T06:13:40.333ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
160992025-04-30T06:13:40.333ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161002025-04-30T06:13:40.340ZDEBGomicron-dev (BackgroundTasks): activating background_task = blueprint_loader iteration = 5 reason = Timeout
161012025-04-30T06:13:40.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig
161022025-04-30T06:13:40.340ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161032025-04-30T06:13:40.340ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
161042025-04-30T06:13:40.340ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
161052025-04-30T06:13:40.345ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
161062025-04-30T06:13:40.350ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161072025-04-30T06:13:40.360ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
161082025-04-30T06:13:40.362ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = BlueprintConfig result = Ok(())
161092025-04-30T06:13:40.362ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
161102025-04-30T06:13:40.362ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
161112025-04-30T06:13:40.364ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
161122025-04-30T06:13:40.364ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
161132025-04-30T06:13:40.364ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
161142025-04-30T06:13:40.364ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 351.90114ms iteration = 60 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
161152025-04-30T06:13:40.364ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 61 reason = Signaled
161162025-04-30T06:13:40.364ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
161172025-04-30T06:13:40.364ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161182025-04-30T06:13:40.364ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
161192025-04-30T06:13:40.364ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
161202025-04-30T06:13:40.366ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161212025-04-30T06:13:40.367ZTRAComicron-dev (BackgroundTasks): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bfc4639-5ca6-4323-a678-d4a0347a6fdf, lookup_type: ById(1bfc4639-5ca6-4323-a678-d4a0347a6fdf) }
161222025-04-30T06:13:40.367ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = blueprint_loader resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161232025-04-30T06:13:40.367ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
161242025-04-30T06:13:40.367ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
161252025-04-30T06:13:40.368ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161262025-04-30T06:13:40.369ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
161272025-04-30T06:13:40.374ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
161282025-04-30T06:13:40.374ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
161292025-04-30T06:13:40.374ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
161302025-04-30T06:13:40.380ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
161312025-04-30T06:13:40.383ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161322025-04-30T06:13:40.385ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
161332025-04-30T06:13:40.385ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
161342025-04-30T06:13:40.386ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 6f 36 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 o6..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:57699 req_id = e1e8bdd8-aa21-4e98-a900-da4e1144eaf8
161352025-04-30T06:13:40.386ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 28470,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57699 req_id = e1e8bdd8-aa21-4e98-a900-da4e1144eaf8
161362025-04-30T06:13:40.387ZDEBGomicron-dev (store): query key key = _mgs._tcp
161372025-04-30T06:13:40.387ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
161382025-04-30T06:13:40.387ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:57699 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = e1e8bdd8-aa21-4e98-a900-da4e1144eaf8
161392025-04-30T06:13:40.388ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2485, tv_nsec: 186913192 } })
161402025-04-30T06:13:40.388ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
161412025-04-30T06:13:40.389ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: f2 0b 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:51203 req_id = 1d1a1f9c-0d35-4ca9-88cc-49aa0bfda071
161422025-04-30T06:13:40.389ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 61963,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51203 req_id = 1d1a1f9c-0d35-4ca9-88cc-49aa0bfda071
161432025-04-30T06:13:40.389ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
161442025-04-30T06:13:40.389ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:51203 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1d1a1f9c-0d35-4ca9-88cc-49aa0bfda071
161452025-04-30T06:13:40.396ZDEBGomicron-dev (BackgroundTasks): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Blueprint { parent: Fleet, key: 1bfc4639-5ca6-4323-a678-d4a0347a6fdf, lookup_type: ById(1bfc4639-5ca6-4323-a678-d4a0347a6fdf) } result = Ok(())
161462025-04-30T06:13:40.396ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database
161472025-04-30T06:13:40.396ZDEBGomicron-dev (BackgroundTasks): roles background_task = blueprint_loader roles = RoleSet { roles: {} }
161482025-04-30T06:13:40.397ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = blueprint_loader resource = Database result = Ok(())
161492025-04-30T06:13:40.412ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
161502025-04-30T06:13:40.412ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161512025-04-30T06:13:40.412ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
161522025-04-30T06:13:40.412ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
161532025-04-30T06:13:40.413ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
161542025-04-30T06:13:40.423ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161552025-04-30T06:13:40.435ZDEBGomicron-dev (BackgroundTasks): found latest target blueprint (unchanged) background_task = blueprint_loader original_target_id = 1bfc4639-5ca6-4323-a678-d4a0347a6fdf original_time_created = 2025-04-30 06:13:06.783824 UTC target_id = 1bfc4639-5ca6-4323-a678-d4a0347a6fdf time_created = 2025-04-30 06:13:06.783824 UTC
161562025-04-30T06:13:40.436ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = blueprint_loader elapsed = 95.838352ms iteration = 5 status = {"enabled":false,"status":"target blueprint unchanged","target_id":"1bfc4639-5ca6-4323-a678-d4a0347a6fdf","time_created":"2025-04-30T06:13:06.783824Z"}
161572025-04-30T06:13:40.436ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
161582025-04-30T06:13:40.437ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
161592025-04-30T06:13:40.437ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
161602025-04-30T06:13:40.437ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
161612025-04-30T06:13:40.440ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
161622025-04-30T06:13:40.440ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
161632025-04-30T06:13:40.440ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
161642025-04-30T06:13:40.467ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
161652025-04-30T06:13:40.471ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
161662025-04-30T06:13:40.472ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 13 cb 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c ............._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:64484 req_id = 7acc7f17-1143-4010-92ab-eafba825455e
161672025-04-30T06:13:40.472ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 5067,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64484 req_id = 7acc7f17-1143-4010-92ab-eafba825455e
161682025-04-30T06:13:40.472ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
161692025-04-30T06:13:40.473ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 5067, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:64484 req_id = 7acc7f17-1143-4010-92ab-eafba825455e
161702025-04-30T06:13:40.473ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
161712025-04-30T06:13:40.473ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
161722025-04-30T06:13:40.473ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
161732025-04-30T06:13:40.476ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "f661f7b6-dd5d-4c7a-bcc2-0dae50a32fad", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
161742025-04-30T06:13:40.476ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
161752025-04-30T06:13:40.478ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d9c67ef8-255a-4f36-aa44-6f82af5b217a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
161762025-04-30T06:13:40.478ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
161772025-04-30T06:13:40.483ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c2b98eb0-c3e9-4421-af91-83222162eafe", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
161782025-04-30T06:13:40.483ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
161792025-04-30T06:13:40.486ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8645566d-c02f-41c6-85c1-fd0bd3c6c446", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
161802025-04-30T06:13:40.487ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
161812025-04-30T06:13:40.488ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a931f88c-e0b9-4627-8b83-ccabac3f5fd0", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
161822025-04-30T06:13:40.489ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
161832025-04-30T06:13:40.499ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "95ae44ee-7404-45b4-acf1-b29992804216", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
161842025-04-30T06:13:40.500ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
161852025-04-30T06:13:40.505ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "967c1ced-236d-4e86-a862-bc182eb1f902", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
161862025-04-30T06:13:40.505ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
161872025-04-30T06:13:40.520ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4c07cb94-3cab-4928-90d0-38f3ab9627ab", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
161882025-04-30T06:13:40.521ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
161892025-04-30T06:13:40.522ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
161902025-04-30T06:13:40.522ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
161912025-04-30T06:13:40.522ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
161922025-04-30T06:13:40.522ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
161932025-04-30T06:13:40.524ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
161942025-04-30T06:13:40.525ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e3376a75-1cf2-46b3-b074-56b2643ac28e", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
161952025-04-30T06:13:40.525ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
161962025-04-30T06:13:40.527ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
161972025-04-30T06:13:40.535ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
161982025-04-30T06:13:40.535ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
161992025-04-30T06:13:40.535ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
162002025-04-30T06:13:40.536ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162012025-04-30T06:13:40.536ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0c8327eb-bd81-464e-8e06-fe4e4c19b4e0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162022025-04-30T06:13:40.537ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
162032025-04-30T06:13:40.543ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "58681d82-d3bd-4919-9ba5-7dc293a81220", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162042025-04-30T06:13:40.543ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
162052025-04-30T06:13:40.543ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b094175c-9cac-4a39-8561-acae81af0c46", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162062025-04-30T06:13:40.543ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
162072025-04-30T06:13:40.543ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
162082025-04-30T06:13:40.543ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
162092025-04-30T06:13:40.543ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162102025-04-30T06:13:40.543ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea7c48f8-2116-459e-b164-6e15a865022f", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162112025-04-30T06:13:40.543ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
162122025-04-30T06:13:40.550ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d7f8d88a-fe12-404a-8eec-0fd097e3d62a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162132025-04-30T06:13:40.552ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
162142025-04-30T06:13:40.558ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "92319c0b-493b-4362-adc6-684d3e36acb1", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162152025-04-30T06:13:40.558ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
162162025-04-30T06:13:40.562ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "707b195e-7022-4019-ae6c-74abaedb6a38", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162172025-04-30T06:13:40.564ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
162182025-04-30T06:13:40.571ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a589ee74-3e18-4f75-a692-8d8553e4f44c", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162192025-04-30T06:13:40.571ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
162202025-04-30T06:13:40.576ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "109b11f4-3126-443d-b2f4-3ebad11d792d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162212025-04-30T06:13:40.577ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
162222025-04-30T06:13:40.578ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
162232025-04-30T06:13:40.581ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "36d1ff2d-b760-4bb4-ade1-f9551501b597", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162242025-04-30T06:13:40.581ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
162252025-04-30T06:13:40.584ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7222d2fb-1f87-49ac-bf20-cc204cd61034", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162262025-04-30T06:13:40.584ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
162272025-04-30T06:13:40.585ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b640a67-187c-4032-9c07-42bb4095290d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162282025-04-30T06:13:40.585ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
162292025-04-30T06:13:40.587ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "76cbe740-1383-437d-aadd-fb6f8f852694", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162302025-04-30T06:13:40.587ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
162312025-04-30T06:13:40.591ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "88ff13fa-6a82-4ed4-bdab-a855387f5a4f", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162322025-04-30T06:13:40.591ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
162332025-04-30T06:13:40.598ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b371f2d-153b-4b73-9754-220ee5e970d9", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162342025-04-30T06:13:40.598ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
162352025-04-30T06:13:40.603ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5112dc66-ebb6-4450-baaa-87207dabad20", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:39 GMT"} })
162362025-04-30T06:13:40.603ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
162372025-04-30T06:13:40.612ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c48796f1-9c9f-470c-8622-053ec452292a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162382025-04-30T06:13:40.612ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
162392025-04-30T06:13:40.615ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7caf786c-78ef-4f52-89a6-6351e66874fc", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162402025-04-30T06:13:40.615ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
162412025-04-30T06:13:40.617ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "aabf42d6-264e-405d-9381-10e9cd19035e", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162422025-04-30T06:13:40.617ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
162432025-04-30T06:13:40.619ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9d4c88d-84fc-49e3-aa0e-4c0e96bc467c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162442025-04-30T06:13:40.619ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
162452025-04-30T06:13:40.621ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d548158-d680-4205-a28c-208eb07036a2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162462025-04-30T06:13:40.621ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
162472025-04-30T06:13:40.622ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cd779848-5c05-46fd-99e8-14d052c154bc", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162482025-04-30T06:13:40.622ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
162492025-04-30T06:13:40.626ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "328c279d-291d-43b9-9b84-7d94372b8fb5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162502025-04-30T06:13:40.626ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
162512025-04-30T06:13:40.627ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "34215c69-e235-46e7-8f23-7c97a37ba0bb", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162522025-04-30T06:13:40.627ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
162532025-04-30T06:13:40.628ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
162542025-04-30T06:13:40.628ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
162552025-04-30T06:13:40.628ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
162562025-04-30T06:13:40.628ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
162572025-04-30T06:13:40.629ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d1f630f9-c786-4bd7-b9c9-426d782eb1ce", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162582025-04-30T06:13:40.629ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
162592025-04-30T06:13:40.629ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162602025-04-30T06:13:40.637ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162612025-04-30T06:13:40.639ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f7a67641-358d-4b37-b4ee-ce5ce6600687", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162622025-04-30T06:13:40.640ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
162632025-04-30T06:13:40.645ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b89b5cbb-9818-4ce8-9a3f-02fb93ede142", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162642025-04-30T06:13:40.645ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
162652025-04-30T06:13:40.645ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
162662025-04-30T06:13:40.645ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
162672025-04-30T06:13:40.645ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
162682025-04-30T06:13:40.646ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162692025-04-30T06:13:40.647ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc3f504f-12c7-4432-9c30-07735969aea3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162702025-04-30T06:13:40.647ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
162712025-04-30T06:13:40.648ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
162722025-04-30T06:13:40.648ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
162732025-04-30T06:13:40.649ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
162742025-04-30T06:13:40.649ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a87f7def-557e-4369-b9a7-c53f694a7ce7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162752025-04-30T06:13:40.649ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
162762025-04-30T06:13:40.651ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d60a0c6b-5a4b-492d-82a8-073cd7dc29b9", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162772025-04-30T06:13:40.651ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
162782025-04-30T06:13:40.653ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fc69f582-965b-420a-8df1-35014214c2c2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162792025-04-30T06:13:40.654ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
162802025-04-30T06:13:40.655ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b5b878d-20d3-4c4b-b28a-7e4a8c28777c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162812025-04-30T06:13:40.655ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
162822025-04-30T06:13:40.656ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ecc48752-d2f5-4305-9c76-c56d19229907", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162832025-04-30T06:13:40.656ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
162842025-04-30T06:13:40.657ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0d56f8e6-03a7-4a4b-bc16-4479cb4a09cc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162852025-04-30T06:13:40.657ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
162862025-04-30T06:13:40.659ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "efbc0d20-3db4-4f78-ad21-ee604c602491", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162872025-04-30T06:13:40.659ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
162882025-04-30T06:13:40.660ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "bc172834-8bb1-4929-8f57-7e17ad63d9e7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
162892025-04-30T06:13:40.661ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
162902025-04-30T06:13:40.661ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
162912025-04-30T06:13:40.661ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
162922025-04-30T06:13:40.661ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
162932025-04-30T06:13:40.662ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
162942025-04-30T06:13:40.664ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
162952025-04-30T06:13:40.669ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
162962025-04-30T06:13:40.670ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
162972025-04-30T06:13:40.670ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
162982025-04-30T06:13:40.670ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
162992025-04-30T06:13:40.670ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163002025-04-30T06:13:40.670ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163012025-04-30T06:13:40.676ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
163022025-04-30T06:13:40.676ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163032025-04-30T06:13:40.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
163042025-04-30T06:13:40.680ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
163052025-04-30T06:13:40.680ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
163062025-04-30T06:13:40.680ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163072025-04-30T06:13:40.680ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bb9ba67a-cf9d-46cb-adae-d3cea3c752cb uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
163082025-04-30T06:13:40.680ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163092025-04-30T06:13:40.681ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bb9ba67a-cf9d-46cb-adae-d3cea3c752cb resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
163102025-04-30T06:13:40.681ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bb9ba67a-cf9d-46cb-adae-d3cea3c752cb roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
163112025-04-30T06:13:40.681ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163122025-04-30T06:13:40.682ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bb9ba67a-cf9d-46cb-adae-d3cea3c752cb resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
163132025-04-30T06:13:40.687ZINFOomicron-dev (dropshot_internal): request completed latency_us = 6415 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = bb9ba67a-cf9d-46cb-adae-d3cea3c752cb response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
163142025-04-30T06:13:40.687ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb9ba67a-cf9d-46cb-adae-d3cea3c752cb", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
163152025-04-30T06:13:40.687ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
163162025-04-30T06:13:40.688ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9ea3b720-2972-4ab7-a189-a537dca573a1 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
163172025-04-30T06:13:40.688ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9ea3b720-2972-4ab7-a189-a537dca573a1 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
163182025-04-30T06:13:40.688ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9ea3b720-2972-4ab7-a189-a537dca573a1 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
163192025-04-30T06:13:40.689ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9ea3b720-2972-4ab7-a189-a537dca573a1 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
163202025-04-30T06:13:40.727ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
163212025-04-30T06:13:40.727ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
163222025-04-30T06:13:40.727ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
163232025-04-30T06:13:40.727ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
163242025-04-30T06:13:40.727ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
163252025-04-30T06:13:40.727ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
163262025-04-30T06:13:40.728ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
163272025-04-30T06:13:40.729ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
163282025-04-30T06:13:40.729ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
163292025-04-30T06:13:40.729ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
163302025-04-30T06:13:40.730ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
163312025-04-30T06:13:40.736ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
163322025-04-30T06:13:40.736ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
163332025-04-30T06:13:40.739ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:62199 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
163342025-04-30T06:13:40.747ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:62199 req_id = e306dd26-7b78-45d8-91f9-42aaabd10eb6 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
163352025-04-30T06:13:40.747ZINFOomicron-dev (dropshot): request completed latency_us = 558 local_addr = [::1]:61881 method = GET remote_addr = [::1]:62199 req_id = e306dd26-7b78-45d8-91f9-42aaabd10eb6 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
163362025-04-30T06:13:40.748ZINFOomicron-dev (dropshot_internal): request completed latency_us = 60487 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 9ea3b720-2972-4ab7-a189-a537dca573a1 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
163372025-04-30T06:13:40.752ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "e306dd26-7b78-45d8-91f9-42aaabd10eb6", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
163382025-04-30T06:13:40.752ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163392025-04-30T06:13:40.755ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "9ea3b720-2972-4ab7-a189-a537dca573a1", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
163402025-04-30T06:13:40.756ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
163412025-04-30T06:13:40.756ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
163422025-04-30T06:13:40.756ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
163432025-04-30T06:13:40.758ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
163442025-04-30T06:13:40.758ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
163452025-04-30T06:13:40.758ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
163462025-04-30T06:13:40.760ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
163472025-04-30T06:13:40.760ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
163482025-04-30T06:13:40.760ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
163492025-04-30T06:13:40.761ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
163502025-04-30T06:13:40.761ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
163512025-04-30T06:13:40.761ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
163522025-04-30T06:13:40.761ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163532025-04-30T06:13:40.761ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163542025-04-30T06:13:40.762ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163552025-04-30T06:13:40.764ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
163562025-04-30T06:13:40.764ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
163572025-04-30T06:13:40.765ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
163582025-04-30T06:13:40.765ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163592025-04-30T06:13:40.770ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
163602025-04-30T06:13:40.771ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163612025-04-30T06:13:40.771ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163622025-04-30T06:13:40.772ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163632025-04-30T06:13:40.799ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
163642025-04-30T06:13:40.800ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
163652025-04-30T06:13:40.801ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
163662025-04-30T06:13:40.801ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 436.385028ms iteration = 61 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
163672025-04-30T06:13:40.801ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 62 reason = Signaled
163682025-04-30T06:13:40.801ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
163692025-04-30T06:13:40.801ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
163702025-04-30T06:13:40.801ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163712025-04-30T06:13:40.801ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163722025-04-30T06:13:40.802ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163732025-04-30T06:13:40.804ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163742025-04-30T06:13:40.811ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
163752025-04-30T06:13:40.812ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
163762025-04-30T06:13:40.812ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
163772025-04-30T06:13:40.812ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
163782025-04-30T06:13:40.816ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
163792025-04-30T06:13:40.816ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
163802025-04-30T06:13:40.818ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 5f a4 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 _............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:55673 req_id = 5d3f493d-6452-4b77-8cbf-8f43d591892c
163812025-04-30T06:13:40.818ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24484,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55673 req_id = 5d3f493d-6452-4b77-8cbf-8f43d591892c
163822025-04-30T06:13:40.818ZDEBGomicron-dev (store): query key key = _mgs._tcp
163832025-04-30T06:13:40.818ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
163842025-04-30T06:13:40.818ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:55673 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5d3f493d-6452-4b77-8cbf-8f43d591892c
163852025-04-30T06:13:40.819ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2485, tv_nsec: 618089433 } })
163862025-04-30T06:13:40.819ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
163872025-04-30T06:13:40.819ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 3b 8f 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ;...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:55676 req_id = 0c7f440a-10e9-4d55-8e91-5b4fae9b60e9
163882025-04-30T06:13:40.819ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 15247,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:55676 req_id = 0c7f440a-10e9-4d55-8e91-5b4fae9b60e9
163892025-04-30T06:13:40.819ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
163902025-04-30T06:13:40.820ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:55676 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0c7f440a-10e9-4d55-8e91-5b4fae9b60e9
163912025-04-30T06:13:40.881ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
163922025-04-30T06:13:40.881ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
163932025-04-30T06:13:40.881ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
163942025-04-30T06:13:40.881ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
163952025-04-30T06:13:40.882ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
163962025-04-30T06:13:40.887ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
163972025-04-30T06:13:40.894ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
163982025-04-30T06:13:40.895ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
163992025-04-30T06:13:40.895ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
164002025-04-30T06:13:40.895ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
164012025-04-30T06:13:40.901ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
164022025-04-30T06:13:40.903ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 67 c4 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c g............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:62849 req_id = 6294931a-da93-4bef-b923-d25ed8d6bae1
164032025-04-30T06:13:40.903ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26564,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62849 req_id = 6294931a-da93-4bef-b923-d25ed8d6bae1
164042025-04-30T06:13:40.903ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
164052025-04-30T06:13:40.903ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 26564, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:62849 req_id = 6294931a-da93-4bef-b923-d25ed8d6bae1
164062025-04-30T06:13:40.903ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
164072025-04-30T06:13:40.903ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
164082025-04-30T06:13:40.903ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
164092025-04-30T06:13:40.904ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
164102025-04-30T06:13:40.904ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
164112025-04-30T06:13:40.904ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
164122025-04-30T06:13:40.905ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f7f8449-4f56-402b-80ea-1a1b3c0f31c1", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164132025-04-30T06:13:40.906ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
164142025-04-30T06:13:40.908ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "94ddad28-ac80-46c2-9f7b-1a3eda741183", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164152025-04-30T06:13:40.909ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
164162025-04-30T06:13:40.914ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8bf9ed3e-e0d5-443c-88a9-0fa4bc8d6d21", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164172025-04-30T06:13:40.914ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
164182025-04-30T06:13:40.916ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e0a76781-1bde-4419-8862-107e81534fcb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164192025-04-30T06:13:40.916ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
164202025-04-30T06:13:40.918ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "44f31fe0-a806-42c1-b063-06de30ac71aa", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164212025-04-30T06:13:40.925ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
164222025-04-30T06:13:40.927ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd1beb6d-abe7-4773-9dca-b51b370b9dd8", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164232025-04-30T06:13:40.927ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
164242025-04-30T06:13:40.929ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ba3cb50-dc4f-4a5f-9020-8c57ac5fcc0f", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164252025-04-30T06:13:40.929ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
164262025-04-30T06:13:40.931ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d16e0c8b-d1ea-48ca-af93-94ef4c251f57", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164272025-04-30T06:13:40.932ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
164282025-04-30T06:13:40.933ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c52bf8be-664d-41f2-a9b5-22aea6e06e54", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164292025-04-30T06:13:40.933ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
164302025-04-30T06:13:40.938ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d18299b-f743-4154-ae75-9563641298e2", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164312025-04-30T06:13:40.938ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
164322025-04-30T06:13:40.940ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ca8c10fe-1c4e-495c-93ea-5be2cc0a872e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164332025-04-30T06:13:40.940ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
164342025-04-30T06:13:40.940ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c87a1db1-830b-4d84-976d-fce1b0977bd2", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164352025-04-30T06:13:40.940ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
164362025-04-30T06:13:40.942ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f4079d0-6129-4ab4-af6e-130d6a28f8ae", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164372025-04-30T06:13:40.942ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
164382025-04-30T06:13:40.944ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "596bc848-f05c-4824-aa81-3e5b0ff12482", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164392025-04-30T06:13:40.944ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
164402025-04-30T06:13:40.946ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4741c9a7-8acf-437e-a6f6-5f9462647558", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164412025-04-30T06:13:40.946ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
164422025-04-30T06:13:40.951ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "cb969c79-f940-4a16-ba3a-9d6205bc84fc", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164432025-04-30T06:13:40.952ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
164442025-04-30T06:13:40.968ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6b02c94b-41f9-496a-a002-cf5c1c718a81", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164452025-04-30T06:13:40.968ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
164462025-04-30T06:13:40.981ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "61de9571-c728-4626-b9d4-13442dcbc0f0", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164472025-04-30T06:13:40.984ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
164482025-04-30T06:13:40.993ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "dd50690a-8ff0-4a49-b085-10cc4a17123a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164492025-04-30T06:13:40.994ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
164502025-04-30T06:13:40.996ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0b2d000e-3b9d-41ab-b3e9-f318db91f165", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164512025-04-30T06:13:40.997ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
164522025-04-30T06:13:40.998ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "43dd39b7-f97d-494b-98a8-01f1c444a722", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164532025-04-30T06:13:40.998ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
164542025-04-30T06:13:41.007ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e1828ac3-283d-427d-8c37-b0aca77e16c9", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164552025-04-30T06:13:41.008ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
164562025-04-30T06:13:41.009ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "255bad29-eedc-4ba0-925e-0223406e36c4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164572025-04-30T06:13:41.009ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
164582025-04-30T06:13:41.011ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d5536b8c-274a-4550-9f04-fb187bdbe407", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164592025-04-30T06:13:41.011ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
164602025-04-30T06:13:41.013ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5df3d641-89de-412d-8faf-71874bb53c2f", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164612025-04-30T06:13:41.013ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
164622025-04-30T06:13:41.016ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc95e7bb-76fa-4a08-8f12-88f5a632c3dd", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164632025-04-30T06:13:41.016ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
164642025-04-30T06:13:41.017ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e8cdeae-5c3f-459c-8e5b-7dd8ea8d4a05", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164652025-04-30T06:13:41.018ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
164662025-04-30T06:13:41.020ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d93c0bcf-5011-4b22-91ca-d1d5587411df", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164672025-04-30T06:13:41.020ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
164682025-04-30T06:13:41.022ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0d93f02-ecc0-42d7-b76a-20185d5608cd", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164692025-04-30T06:13:41.022ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
164702025-04-30T06:13:41.025ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "698a6c7e-ad10-4889-95fb-8da0393a6d82", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164712025-04-30T06:13:41.025ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
164722025-04-30T06:13:41.026ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d64d36e0-91f8-4935-8d74-740a351a7be7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164732025-04-30T06:13:41.026ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
164742025-04-30T06:13:41.027ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9ae0a5d-39b7-4055-98fc-da36f7c40cb0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164752025-04-30T06:13:41.028ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
164762025-04-30T06:13:41.028ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ff2cdc15-a266-46c4-a031-a5b0dfef4961", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164772025-04-30T06:13:41.029ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
164782025-04-30T06:13:41.029ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "52cc5234-4806-4ea4-a794-0f021fc4d531", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164792025-04-30T06:13:41.030ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
164802025-04-30T06:13:41.032ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f53fb098-5d24-46f6-9da7-af2501de2bcb", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164812025-04-30T06:13:41.032ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
164822025-04-30T06:13:41.035ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a9428ff6-de20-45bb-b4cb-dca436199328", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164832025-04-30T06:13:41.035ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
164842025-04-30T06:13:41.037ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fbd01231-6514-4906-a0a2-e7874a19e517", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164852025-04-30T06:13:41.037ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
164862025-04-30T06:13:41.039ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "14cc7d20-a233-4b42-a9b3-c61b8f5d2836", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164872025-04-30T06:13:41.039ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
164882025-04-30T06:13:41.041ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c36f0352-6c00-4871-bfb5-65d8921c4d89", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164892025-04-30T06:13:41.041ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
164902025-04-30T06:13:41.043ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bab0ffe-4b2d-4c44-83bc-542a810b0e40", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164912025-04-30T06:13:41.043ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
164922025-04-30T06:13:41.044ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3f1cf173-3936-4ef7-9e9e-099fff74dda7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164932025-04-30T06:13:41.044ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
164942025-04-30T06:13:41.045ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ac464a2-34fd-450a-9a2e-5960b484ae39", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164952025-04-30T06:13:41.045ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
164962025-04-30T06:13:41.046ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
164972025-04-30T06:13:41.047ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f7f846a-71c0-49d7-bd44-f1a32abe202b", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
164982025-04-30T06:13:41.048ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
164992025-04-30T06:13:41.049ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "36e70c55-fde7-4d55-95a8-5715d18a8b4d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
165002025-04-30T06:13:41.049ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
165012025-04-30T06:13:41.051ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "91493ee8-8941-4eda-afcf-0f37fa3fa555", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
165022025-04-30T06:13:41.051ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
165032025-04-30T06:13:41.051ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165042025-04-30T06:13:41.051ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165052025-04-30T06:13:41.051ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165062025-04-30T06:13:41.052ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165072025-04-30T06:13:41.059ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165082025-04-30T06:13:41.066ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
165092025-04-30T06:13:41.066ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
165102025-04-30T06:13:41.066ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165112025-04-30T06:13:41.066ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165122025-04-30T06:13:41.066ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165132025-04-30T06:13:41.067ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165142025-04-30T06:13:41.090ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
165152025-04-30T06:13:41.090ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165162025-04-30T06:13:41.093ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
165172025-04-30T06:13:41.093ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
165182025-04-30T06:13:41.094ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
165192025-04-30T06:13:41.099ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
165202025-04-30T06:13:41.100ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
165212025-04-30T06:13:41.102ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
165222025-04-30T06:13:41.102ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165232025-04-30T06:13:41.102ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165242025-04-30T06:13:41.104ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165252025-04-30T06:13:41.105ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
165262025-04-30T06:13:41.105ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
165272025-04-30T06:13:41.107ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
165282025-04-30T06:13:41.107ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165292025-04-30T06:13:41.107ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165302025-04-30T06:13:41.107ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
165312025-04-30T06:13:41.108ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165322025-04-30T06:13:41.137ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165332025-04-30T06:13:41.143ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
165342025-04-30T06:13:41.143ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165352025-04-30T06:13:41.143ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
165362025-04-30T06:13:41.145ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165372025-04-30T06:13:41.154ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165382025-04-30T06:13:41.154ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
165392025-04-30T06:13:41.154ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165402025-04-30T06:13:41.183ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
165412025-04-30T06:13:41.194ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
165422025-04-30T06:13:41.223ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
165432025-04-30T06:13:41.224ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
165442025-04-30T06:13:41.224ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:56201 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
165452025-04-30T06:13:41.225ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:56201 req_id = 6e548b64-e824-47e5-85a2-839363045ba7 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
165462025-04-30T06:13:41.225ZINFOomicron-dev (dropshot): request completed latency_us = 484 local_addr = [::1]:61881 method = GET remote_addr = [::1]:56201 req_id = 6e548b64-e824-47e5-85a2-839363045ba7 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
165472025-04-30T06:13:41.225ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "6e548b64-e824-47e5-85a2-839363045ba7", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
165482025-04-30T06:13:41.229ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
165492025-04-30T06:13:41.229ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
165502025-04-30T06:13:41.229ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
165512025-04-30T06:13:41.231ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
165522025-04-30T06:13:41.231ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165532025-04-30T06:13:41.231ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165542025-04-30T06:13:41.231ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165552025-04-30T06:13:41.231ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165562025-04-30T06:13:41.234ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
165572025-04-30T06:13:41.234ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165582025-04-30T06:13:41.234ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165592025-04-30T06:13:41.234ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
165602025-04-30T06:13:41.235ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165612025-04-30T06:13:41.246ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165622025-04-30T06:13:41.250ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
165632025-04-30T06:13:41.251ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165642025-04-30T06:13:41.251ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165652025-04-30T06:13:41.253ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165662025-04-30T06:13:41.255ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165672025-04-30T06:13:41.261ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
165682025-04-30T06:13:41.262ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165692025-04-30T06:13:41.262ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
165702025-04-30T06:13:41.262ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165712025-04-30T06:13:41.267ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
165722025-04-30T06:13:41.267ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
165732025-04-30T06:13:41.268ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
165742025-04-30T06:13:41.312ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
165752025-04-30T06:13:41.316ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
165762025-04-30T06:13:41.316ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
165772025-04-30T06:13:41.316ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 515.323438ms iteration = 62 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
165782025-04-30T06:13:41.316ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 63 reason = Signaled
165792025-04-30T06:13:41.316ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
165802025-04-30T06:13:41.316ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
165812025-04-30T06:13:41.316ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165822025-04-30T06:13:41.316ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165832025-04-30T06:13:41.320ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165842025-04-30T06:13:41.330ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
165852025-04-30T06:13:41.334ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
165862025-04-30T06:13:41.334ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
165872025-04-30T06:13:41.334ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
165882025-04-30T06:13:41.334ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
165892025-04-30T06:13:41.338ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
165902025-04-30T06:13:41.338ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
165912025-04-30T06:13:41.339ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: d3 c3 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 ............._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:49463 req_id = 63f6ab49-65f5-4317-8557-ef2ebe89c4c8
165922025-04-30T06:13:41.339ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 54211,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:49463 req_id = 63f6ab49-65f5-4317-8557-ef2ebe89c4c8
165932025-04-30T06:13:41.340ZDEBGomicron-dev (store): query key key = _mgs._tcp
165942025-04-30T06:13:41.340ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
165952025-04-30T06:13:41.341ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:49463 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 63f6ab49-65f5-4317-8557-ef2ebe89c4c8
165962025-04-30T06:13:41.342ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2486, tv_nsec: 141103585 } })
165972025-04-30T06:13:41.342ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
165982025-04-30T06:13:41.342ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: c0 c1 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:64359 req_id = 9632346a-2b52-4945-a60a-ca1c68dc81cf
165992025-04-30T06:13:41.342ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 49345,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64359 req_id = 9632346a-2b52-4945-a60a-ca1c68dc81cf
166002025-04-30T06:13:41.343ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
166012025-04-30T06:13:41.343ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:64359 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9632346a-2b52-4945-a60a-ca1c68dc81cf
166022025-04-30T06:13:41.453ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
166032025-04-30T06:13:41.454ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
166042025-04-30T06:13:41.454ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
166052025-04-30T06:13:41.454ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
166062025-04-30T06:13:41.454ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
166072025-04-30T06:13:41.454ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 7b 3d 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c {=..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:58923 req_id = ffe7aa5e-e984-41bc-b52e-2dd1ef562cf7
166082025-04-30T06:13:41.455ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 31549,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58923 req_id = ffe7aa5e-e984-41bc-b52e-2dd1ef562cf7
166092025-04-30T06:13:41.455ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
166102025-04-30T06:13:41.455ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
166112025-04-30T06:13:41.455ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 31549, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:58923 req_id = ffe7aa5e-e984-41bc-b52e-2dd1ef562cf7
166122025-04-30T06:13:41.455ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
166132025-04-30T06:13:41.455ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
166142025-04-30T06:13:41.455ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
166152025-04-30T06:13:41.458ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "45a4cee1-8f90-48f9-a6b9-9abbbbe8e69b", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166162025-04-30T06:13:41.458ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
166172025-04-30T06:13:41.458ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
166182025-04-30T06:13:41.460ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa08ce90-9f96-43af-8961-ea37fb863537", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166192025-04-30T06:13:41.460ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
166202025-04-30T06:13:41.462ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "dadc19d4-15b6-4920-b5b9-10f9bbbeafc0", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166212025-04-30T06:13:41.462ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
166222025-04-30T06:13:41.463ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
166232025-04-30T06:13:41.463ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
166242025-04-30T06:13:41.463ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
166252025-04-30T06:13:41.464ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
166262025-04-30T06:13:41.464ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fdda30b-f8d9-4c4d-b01e-08cae77bf7fb", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166272025-04-30T06:13:41.464ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
166282025-04-30T06:13:41.466ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7521940-dedc-4e81-8a3e-9c4fa175b767", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166292025-04-30T06:13:41.466ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
166302025-04-30T06:13:41.467ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
166312025-04-30T06:13:41.467ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
166322025-04-30T06:13:41.468ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
166332025-04-30T06:13:41.468ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b48beeb7-1d59-49f8-b922-a659738cbdb7", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166342025-04-30T06:13:41.468ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
166352025-04-30T06:13:41.470ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b788b91d-f7b0-42a9-a182-2f0bd0a2e6ec", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166362025-04-30T06:13:41.470ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
166372025-04-30T06:13:41.472ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e94b424c-81da-48e5-9c8c-04619f3e8db7", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166382025-04-30T06:13:41.472ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
166392025-04-30T06:13:41.473ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "03ccb764-0ee7-4cd2-947a-14e24ad3435b", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166402025-04-30T06:13:41.473ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
166412025-04-30T06:13:41.476ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "67cd5afa-8a61-4d6c-aef9-0c705ce23ec8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166422025-04-30T06:13:41.477ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
166432025-04-30T06:13:41.478ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2b870cc0-b70d-4b7b-be29-59336389941e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166442025-04-30T06:13:41.478ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
166452025-04-30T06:13:41.479ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f8e791ca-9fcd-4890-a4e2-4858a8120f0e", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166462025-04-30T06:13:41.479ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
166472025-04-30T06:13:41.481ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd87d11b-42f8-4d03-b0e3-ca784e687c53", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166482025-04-30T06:13:41.481ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
166492025-04-30T06:13:41.484ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "12512fae-cf6b-4870-b37a-5cefc510101e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166502025-04-30T06:13:41.484ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
166512025-04-30T06:13:41.485ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fba1a8c0-ac6b-47e0-8e56-7889fdded6ee", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166522025-04-30T06:13:41.486ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
166532025-04-30T06:13:41.488ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "5a5c0b3e-7208-4bf3-9aa9-051538f9b9e4", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166542025-04-30T06:13:41.488ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
166552025-04-30T06:13:41.490ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "541453b2-9b35-42e0-a44f-717b7717adbb", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166562025-04-30T06:13:41.490ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
166572025-04-30T06:13:41.492ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "0db9fbee-d3fa-40d8-8862-76163c10ac7a", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166582025-04-30T06:13:41.492ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
166592025-04-30T06:13:41.494ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5184287-b729-48a2-a671-48477e7f2a4d", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166602025-04-30T06:13:41.494ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
166612025-04-30T06:13:41.495ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1d4a61d9-b4c7-4f08-8b98-f7e7f2b05a99", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166622025-04-30T06:13:41.495ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
166632025-04-30T06:13:41.496ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d49c7f6-2198-415a-aa41-8bfdafa4b62d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166642025-04-30T06:13:41.496ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
166652025-04-30T06:13:41.498ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "2c1c8fc3-b30a-44c6-83d1-edaa011df9a7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166662025-04-30T06:13:41.498ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
166672025-04-30T06:13:41.498ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
166682025-04-30T06:13:41.500ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea007834-36c2-4984-a513-1e90fa17dfe4", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166692025-04-30T06:13:41.500ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
166702025-04-30T06:13:41.501ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "560dd084-261b-4717-b58b-5f1d0764e55a", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166712025-04-30T06:13:41.501ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
166722025-04-30T06:13:41.503ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "396cbf44-6fae-4a07-9ff8-657051865c45", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166732025-04-30T06:13:41.503ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
166742025-04-30T06:13:41.505ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "88810ea2-00b6-4400-a39d-9a19ef64d4d3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166752025-04-30T06:13:41.505ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
166762025-04-30T06:13:41.507ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3cb0e201-e767-440d-9aaf-0b322d5a8925", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166772025-04-30T06:13:41.507ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
166782025-04-30T06:13:41.509ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1fdaeaa6-51f6-4742-928d-4ae4f610ab06", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166792025-04-30T06:13:41.509ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
166802025-04-30T06:13:41.511ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a1f9e4da-cf23-4a78-945e-2b6214b47278", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166812025-04-30T06:13:41.511ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
166822025-04-30T06:13:41.513ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e74c7575-8c52-4613-8d13-2b5ba15c4c1b", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166832025-04-30T06:13:41.513ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
166842025-04-30T06:13:41.514ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "513b7631-0245-4a8f-998c-14b8a38b076a", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166852025-04-30T06:13:41.514ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
166862025-04-30T06:13:41.515ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ca75556-114a-4e00-9734-32a22f79ef18", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166872025-04-30T06:13:41.516ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
166882025-04-30T06:13:41.517ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3496cdda-772f-4229-9f64-b7c3c0951743", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166892025-04-30T06:13:41.517ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
166902025-04-30T06:13:41.518ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d71f1a60-4582-4f05-b259-00a2ed08de1c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166912025-04-30T06:13:41.518ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
166922025-04-30T06:13:41.519ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd7ce61d-4c54-4976-8899-be369cf805ef", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166932025-04-30T06:13:41.519ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
166942025-04-30T06:13:41.521ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c0eb23e0-f448-41ba-9247-c5b0a659b6d0", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166952025-04-30T06:13:41.521ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
166962025-04-30T06:13:41.523ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e8bff91c-8f8c-48c3-869d-17378666e57b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166972025-04-30T06:13:41.523ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
166982025-04-30T06:13:41.526ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3238ffa0-5cb7-4c37-aba8-ce93ad16c1c5", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
166992025-04-30T06:13:41.527ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
167002025-04-30T06:13:41.529ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c9afcf1d-75f9-4654-95a4-cab391cad391", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
167012025-04-30T06:13:41.529ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
167022025-04-30T06:13:41.531ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f081cbab-f26f-466f-8357-a60e3e4b71d2", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
167032025-04-30T06:13:41.531ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
167042025-04-30T06:13:41.535ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "51ccb913-a32d-454b-b590-2c10576a2020", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
167052025-04-30T06:13:41.535ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
167062025-04-30T06:13:41.536ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b2ff4e88-fac5-4aaa-ba29-663a592eb84f", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
167072025-04-30T06:13:41.536ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
167082025-04-30T06:13:41.537ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d580c43e-266f-47f3-8479-a68a006ae3d7", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
167092025-04-30T06:13:41.537ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
167102025-04-30T06:13:41.539ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e9888658-4842-4473-89e7-be834c6ba749", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
167112025-04-30T06:13:41.539ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
167122025-04-30T06:13:41.539ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4ac4d944-7b33-42be-b0eb-88ad2c4867bc", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:40 GMT"} })
167132025-04-30T06:13:41.539ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
167142025-04-30T06:13:41.539ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
167152025-04-30T06:13:41.540ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
167162025-04-30T06:13:41.540ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
167172025-04-30T06:13:41.540ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
167182025-04-30T06:13:41.549ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
167192025-04-30T06:13:41.549ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
167202025-04-30T06:13:41.550ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
167212025-04-30T06:13:41.550ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
167222025-04-30T06:13:41.554ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
167232025-04-30T06:13:41.567ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167242025-04-30T06:13:41.568ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167252025-04-30T06:13:41.574ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
167262025-04-30T06:13:41.574ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
167272025-04-30T06:13:41.574ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
167282025-04-30T06:13:41.574ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
167292025-04-30T06:13:41.574ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
167302025-04-30T06:13:41.575ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
167312025-04-30T06:13:41.576ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
167322025-04-30T06:13:41.576ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
167332025-04-30T06:13:41.576ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
167342025-04-30T06:13:41.577ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
167352025-04-30T06:13:41.590ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
167362025-04-30T06:13:41.591ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
167372025-04-30T06:13:41.591ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
167382025-04-30T06:13:41.592ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
167392025-04-30T06:13:41.594ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
167402025-04-30T06:13:41.594ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
167412025-04-30T06:13:41.594ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
167422025-04-30T06:13:41.596ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
167432025-04-30T06:13:41.596ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
167442025-04-30T06:13:41.601ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 26 b0 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 &............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:58186 req_id = cbe95850-885b-4300-bda9-30bafc286665
167452025-04-30T06:13:41.601ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 9904,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58186 req_id = cbe95850-885b-4300-bda9-30bafc286665
167462025-04-30T06:13:41.601ZDEBGomicron-dev (store): query key key = _dendrite._tcp
167472025-04-30T06:13:41.601ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
167482025-04-30T06:13:41.602ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:58186 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = cbe95850-885b-4300-bda9-30bafc286665
167492025-04-30T06:13:41.602ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2486, tv_nsec: 401886909 } })
167502025-04-30T06:13:41.603ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
167512025-04-30T06:13:41.607ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: a1 78 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .x..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:63023 req_id = 1488f940-cd96-488b-9b3c-a8e5fa2e4da8
167522025-04-30T06:13:41.607ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 41336,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63023 req_id = 1488f940-cd96-488b-9b3c-a8e5fa2e4da8
167532025-04-30T06:13:41.607ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
167542025-04-30T06:13:41.608ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:63023 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 1488f940-cd96-488b-9b3c-a8e5fa2e4da8
167552025-04-30T06:13:41.608ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
167562025-04-30T06:13:41.616ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
167572025-04-30T06:13:41.616ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
167582025-04-30T06:13:41.617ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: ac 52 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 .R..........._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:63191 req_id = 2bc66afe-b6b7-4fa2-b9b9-560c1ffcddbd
167592025-04-30T06:13:41.617ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 25 98 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 %............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:53853 req_id = ccfdab6d-03f2-4452-b67e-59ad2ac34003
167602025-04-30T06:13:41.617ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 9624,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:53853 req_id = ccfdab6d-03f2-4452-b67e-59ad2ac34003
167612025-04-30T06:13:41.617ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 44114,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63191 req_id = 2bc66afe-b6b7-4fa2-b9b9-560c1ffcddbd
167622025-04-30T06:13:41.617ZDEBGomicron-dev (store): query key key = _dendrite._tcp
167632025-04-30T06:13:41.617ZDEBGomicron-dev (store): query key key = _dendrite._tcp
167642025-04-30T06:13:41.617ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
167652025-04-30T06:13:41.617ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63191 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 2bc66afe-b6b7-4fa2-b9b9-560c1ffcddbd
167662025-04-30T06:13:41.618ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
167672025-04-30T06:13:41.618ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:53853 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = ccfdab6d-03f2-4452-b67e-59ad2ac34003
167682025-04-30T06:13:41.620ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2486, tv_nsec: 418364439 } })
167692025-04-30T06:13:41.620ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
167702025-04-30T06:13:41.620ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 47 17 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e G...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:62729 req_id = 91a2390d-f8a1-4e75-b061-b8327e0db028
167712025-04-30T06:13:41.620ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 18199,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62729 req_id = 91a2390d-f8a1-4e75-b061-b8327e0db028
167722025-04-30T06:13:41.620ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
167732025-04-30T06:13:41.620ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:62729 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 91a2390d-f8a1-4e75-b061-b8327e0db028
167742025-04-30T06:13:41.620ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
167752025-04-30T06:13:41.620ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2486, tv_nsec: 419669279 } })
167762025-04-30T06:13:41.620ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
167772025-04-30T06:13:41.621ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 41 cc 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e A...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:50546 req_id = 0a60b36d-7f54-4118-ba4a-6d1ce29ef07e
167782025-04-30T06:13:41.621ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 16844,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:50546 req_id = 0a60b36d-7f54-4118-ba4a-6d1ce29ef07e
167792025-04-30T06:13:41.621ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
167802025-04-30T06:13:41.621ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:50546 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 0a60b36d-7f54-4118-ba4a-6d1ce29ef07e
167812025-04-30T06:13:41.622ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
167822025-04-30T06:13:41.650ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
167832025-04-30T06:13:41.707ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
167842025-04-30T06:13:41.707ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
167852025-04-30T06:13:41.708ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
167862025-04-30T06:13:41.708ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
167872025-04-30T06:13:41.708ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
167882025-04-30T06:13:41.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167892025-04-30T06:13:41.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167902025-04-30T06:13:41.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167912025-04-30T06:13:41.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167922025-04-30T06:13:41.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167932025-04-30T06:13:41.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
167942025-04-30T06:13:41.784ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
167952025-04-30T06:13:41.791ZDEBGomicron-dev (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
167962025-04-30T06:13:41.791ZDEBGomicron-dev (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
167972025-04-30T06:13:41.791ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
167982025-04-30T06:13:41.793ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
167992025-04-30T06:13:41.833ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
168002025-04-30T06:13:41.833ZDEBGomicron-dev (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
168012025-04-30T06:13:41.833ZDEBGomicron-dev (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
168022025-04-30T06:13:41.833ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
168032025-04-30T06:13:41.835ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
168042025-04-30T06:13:41.865ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
168052025-04-30T06:13:41.865ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
168062025-04-30T06:13:41.866ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:42027 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
168072025-04-30T06:13:41.867ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:42027 req_id = bfe77b96-eaac-4c09-aab0-cdce6f696d3a sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
168082025-04-30T06:13:41.867ZINFOomicron-dev (dropshot): request completed latency_us = 475 local_addr = [::1]:61881 method = GET remote_addr = [::1]:42027 req_id = bfe77b96-eaac-4c09-aab0-cdce6f696d3a response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
168092025-04-30T06:13:41.868ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "bfe77b96-eaac-4c09-aab0-cdce6f696d3a", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
168102025-04-30T06:13:41.869ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168112025-04-30T06:13:41.871ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
168122025-04-30T06:13:41.872ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
168132025-04-30T06:13:41.872ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
168142025-04-30T06:13:41.876ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
168152025-04-30T06:13:41.876ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
168162025-04-30T06:13:41.876ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168172025-04-30T06:13:41.877ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168182025-04-30T06:13:41.877ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168192025-04-30T06:13:41.881ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
168202025-04-30T06:13:41.881ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
168212025-04-30T06:13:41.881ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
168222025-04-30T06:13:41.881ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
168232025-04-30T06:13:41.891ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
168242025-04-30T06:13:41.891ZDEBGomicron-dev (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
168252025-04-30T06:13:41.892ZDEBGomicron-dev (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
168262025-04-30T06:13:41.892ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
168272025-04-30T06:13:41.895ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
168282025-04-30T06:13:41.895ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168292025-04-30T06:13:41.917ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
168302025-04-30T06:13:41.918ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168312025-04-30T06:13:41.918ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168322025-04-30T06:13:41.919ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168332025-04-30T06:13:41.972ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
168342025-04-30T06:13:41.972ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
168352025-04-30T06:13:41.974ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
168362025-04-30T06:13:42.061ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
168372025-04-30T06:13:42.062ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
168382025-04-30T06:13:42.062ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 745.676462ms iteration = 63 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
168392025-04-30T06:13:42.062ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 64 reason = Signaled
168402025-04-30T06:13:42.062ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
168412025-04-30T06:13:42.062ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
168422025-04-30T06:13:42.062ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168432025-04-30T06:13:42.062ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168442025-04-30T06:13:42.063ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168452025-04-30T06:13:42.066ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168462025-04-30T06:13:42.070ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
168472025-04-30T06:13:42.070ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
168482025-04-30T06:13:42.070ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
168492025-04-30T06:13:42.071ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
168502025-04-30T06:13:42.090ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
168512025-04-30T06:13:42.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
168522025-04-30T06:13:42.090ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
168532025-04-30T06:13:42.090ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
168542025-04-30T06:13:42.091ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
168552025-04-30T06:13:42.092ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 5b d5 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 [............_mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:52126 req_id = c3675d7e-e54d-4efe-889d-2a6819d56d67
168562025-04-30T06:13:42.092ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23509,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:52126 req_id = c3675d7e-e54d-4efe-889d-2a6819d56d67
168572025-04-30T06:13:42.092ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
168582025-04-30T06:13:42.092ZDEBGomicron-dev (store): query key key = _mgs._tcp
168592025-04-30T06:13:42.093ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
168602025-04-30T06:13:42.093ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:52126 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = c3675d7e-e54d-4efe-889d-2a6819d56d67
168612025-04-30T06:13:42.094ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2486, tv_nsec: 892984068 } })
168622025-04-30T06:13:42.094ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
168632025-04-30T06:13:42.095ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 67 bd 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e g...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:59049 req_id = 3a0db1ec-5447-4329-92ba-6b0d76bfa322
168642025-04-30T06:13:42.095ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 26557,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:59049 req_id = 3a0db1ec-5447-4329-92ba-6b0d76bfa322
168652025-04-30T06:13:42.096ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
168662025-04-30T06:13:42.097ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:59049 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3a0db1ec-5447-4329-92ba-6b0d76bfa322
168672025-04-30T06:13:42.100ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
168682025-04-30T06:13:42.100ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
168692025-04-30T06:13:42.119ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
168702025-04-30T06:13:42.120ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
168712025-04-30T06:13:42.122ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
168722025-04-30T06:13:42.174ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
168732025-04-30T06:13:42.174ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
168742025-04-30T06:13:42.174ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
168752025-04-30T06:13:42.174ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
168762025-04-30T06:13:42.175ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
168772025-04-30T06:13:42.178ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
168782025-04-30T06:13:42.183ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
168792025-04-30T06:13:42.183ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
168802025-04-30T06:13:42.184ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
168812025-04-30T06:13:42.184ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
168822025-04-30T06:13:42.191ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
168832025-04-30T06:13:42.191ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
168842025-04-30T06:13:42.191ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
168852025-04-30T06:13:42.192ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
168862025-04-30T06:13:42.199ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
168872025-04-30T06:13:42.200ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 1f 4b 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .K..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:61667 req_id = 46063a4c-8aec-49ca-afe0-e7b250b6fe5d
168882025-04-30T06:13:42.200ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 8011,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:61667 req_id = 46063a4c-8aec-49ca-afe0-e7b250b6fe5d
168892025-04-30T06:13:42.201ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
168902025-04-30T06:13:42.201ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 8011, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:61667 req_id = 46063a4c-8aec-49ca-afe0-e7b250b6fe5d
168912025-04-30T06:13:42.201ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
168922025-04-30T06:13:42.201ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
168932025-04-30T06:13:42.201ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
168942025-04-30T06:13:42.203ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "eb40fb05-9580-4db0-9fb1-bb498f9dc19d", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
168952025-04-30T06:13:42.203ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
168962025-04-30T06:13:42.205ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "724caf1c-1ec2-4f7c-bf63-f6c3f00745d5", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
168972025-04-30T06:13:42.205ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
168982025-04-30T06:13:42.207ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a2794c8-8c96-4364-8e7b-2da2687f9c91", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
168992025-04-30T06:13:42.207ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
169002025-04-30T06:13:42.209ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b74aa43b-9abe-4f2e-9a84-dccf3e3cf37e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169012025-04-30T06:13:42.209ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
169022025-04-30T06:13:42.211ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "51b76601-9504-415b-ad22-3c03e690a1bd", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169032025-04-30T06:13:42.211ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
169042025-04-30T06:13:42.213ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f91c6897-3141-41a4-aba8-44b0c70b204b", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169052025-04-30T06:13:42.213ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
169062025-04-30T06:13:42.215ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8266a56-13d4-4cff-9d42-c3d491839456", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169072025-04-30T06:13:42.215ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
169082025-04-30T06:13:42.217ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6316feb9-7dfb-417d-b4f0-2ad978fbdf60", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169092025-04-30T06:13:42.217ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
169102025-04-30T06:13:42.217ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "976ce09e-bbf9-4110-bb29-0a43fe51f331", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169112025-04-30T06:13:42.218ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
169122025-04-30T06:13:42.219ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "791e535c-81c8-4718-9e47-7fecc5efe9c5", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169132025-04-30T06:13:42.219ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
169142025-04-30T06:13:42.220ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1f6595a9-f357-4f4c-ab31-79a272c85a3a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169152025-04-30T06:13:42.220ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
169162025-04-30T06:13:42.221ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "464ede32-12bf-44a2-b718-eccd8ad7170b", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169172025-04-30T06:13:42.221ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
169182025-04-30T06:13:42.222ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "fce64649-f765-4929-b4ee-08f3566a22ca", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169192025-04-30T06:13:42.222ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
169202025-04-30T06:13:42.224ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "de4d5d55-ae29-40a2-85b1-c77392b6773a", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169212025-04-30T06:13:42.224ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
169222025-04-30T06:13:42.226ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "575fee93-5683-454c-954e-6516a3d246e2", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169232025-04-30T06:13:42.226ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
169242025-04-30T06:13:42.228ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1a3d817b-84b7-47e2-adef-7d295609f2de", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169252025-04-30T06:13:42.228ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
169262025-04-30T06:13:42.231ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
169272025-04-30T06:13:42.231ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "9c0f4468-c754-47fd-b4c0-533a79c89b78", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169282025-04-30T06:13:42.231ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
169292025-04-30T06:13:42.236ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8f976aca-0ae1-41d2-a942-20c24c50f2e4", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169302025-04-30T06:13:42.236ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
169312025-04-30T06:13:42.239ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad5e2113-4be7-44d8-b81f-c113bc1246fd", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169322025-04-30T06:13:42.239ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
169332025-04-30T06:13:42.240ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ea052a94-c33e-4f60-a64b-1bc55e9bc358", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169342025-04-30T06:13:42.240ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
169352025-04-30T06:13:42.241ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "af2fc89a-acbe-4dc2-a9dc-0e5072199025", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169362025-04-30T06:13:42.242ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
169372025-04-30T06:13:42.244ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fa877a55-d55b-45b6-8bcc-1e306b0f3081", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169382025-04-30T06:13:42.244ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
169392025-04-30T06:13:42.246ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f88b4f1a-74d9-4a84-886c-950a5d4f4cc6", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169402025-04-30T06:13:42.246ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
169412025-04-30T06:13:42.248ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0845fc0-510c-498c-9104-e53692a75726", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169422025-04-30T06:13:42.248ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
169432025-04-30T06:13:42.250ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b9dbcb5-774b-4da8-9bb8-f8ddf1224f9b", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169442025-04-30T06:13:42.250ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
169452025-04-30T06:13:42.252ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "585d3159-d6b4-4818-a8f1-7bff89890f18", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169462025-04-30T06:13:42.252ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
169472025-04-30T06:13:42.254ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b9e528bf-7ff5-462d-b801-2f75e100d1df", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169482025-04-30T06:13:42.254ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
169492025-04-30T06:13:42.257ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cf1a97f-ce9a-479d-9dbc-755b46700db3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169502025-04-30T06:13:42.257ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
169512025-04-30T06:13:42.261ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "34102109-3610-4266-bb8c-8e519b954830", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169522025-04-30T06:13:42.261ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
169532025-04-30T06:13:42.263ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2ca998df-0298-4771-be20-8511d8150ed9", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169542025-04-30T06:13:42.263ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
169552025-04-30T06:13:42.264ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fecea88-5d28-4913-a3d4-adbeb112f5d1", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169562025-04-30T06:13:42.264ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
169572025-04-30T06:13:42.266ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3842924d-7311-4216-a82a-08a0d9743ced", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
169582025-04-30T06:13:42.266ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
169592025-04-30T06:13:42.268ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d9acadd-19bc-4d5a-8e0f-73b6cecc6e1b", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169602025-04-30T06:13:42.269ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
169612025-04-30T06:13:42.271ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "55f3a5e9-efdf-4702-bb8f-2edcf7e8ec19", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169622025-04-30T06:13:42.271ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
169632025-04-30T06:13:42.274ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bf1f9f7f-e9a6-4b4b-b85f-17164b628575", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169642025-04-30T06:13:42.274ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
169652025-04-30T06:13:42.278ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9dce887-8a2b-4039-8bc8-b9ad870ab656", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169662025-04-30T06:13:42.278ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
169672025-04-30T06:13:42.282ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
169682025-04-30T06:13:42.282ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
169692025-04-30T06:13:42.282ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
169702025-04-30T06:13:42.282ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
169712025-04-30T06:13:42.282ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "89d4ceec-3f7a-4d20-8ebe-c4e594048a21", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169722025-04-30T06:13:42.283ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
169732025-04-30T06:13:42.283ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
169742025-04-30T06:13:42.286ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
169752025-04-30T06:13:42.286ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "90b9ad01-c385-4110-af98-77614cf83961", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169762025-04-30T06:13:42.287ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
169772025-04-30T06:13:42.292ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
169782025-04-30T06:13:42.292ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
169792025-04-30T06:13:42.292ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
169802025-04-30T06:13:42.292ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d40d7e64-901c-4ab9-b83a-c0b11b3b947c", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169812025-04-30T06:13:42.292ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
169822025-04-30T06:13:42.293ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
169832025-04-30T06:13:42.296ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
169842025-04-30T06:13:42.296ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
169852025-04-30T06:13:42.296ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
169862025-04-30T06:13:42.303ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "400a01c6-c834-41ad-8d21-f67e5e161fcc", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169872025-04-30T06:13:42.304ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
169882025-04-30T06:13:42.309ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b1fd95d7-d46f-4a9a-8751-de12173b615c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169892025-04-30T06:13:42.309ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
169902025-04-30T06:13:42.310ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "aeb23765-0d30-466e-a9d9-89bbae65b015", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169912025-04-30T06:13:42.310ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
169922025-04-30T06:13:42.312ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "57db5b0d-d1e0-4366-986f-ea333a9650f3", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169932025-04-30T06:13:42.312ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
169942025-04-30T06:13:42.314ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "8cc861d8-bfb1-4a86-bfaa-dbc87b66c8f3", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169952025-04-30T06:13:42.314ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
169962025-04-30T06:13:42.316ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "22e9260b-98a1-470a-a566-868983f18c65", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
169972025-04-30T06:13:42.316ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
169982025-04-30T06:13:42.316ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
169992025-04-30T06:13:42.316ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170002025-04-30T06:13:42.316ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170012025-04-30T06:13:42.318ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170022025-04-30T06:13:42.320ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170032025-04-30T06:13:42.326ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
170042025-04-30T06:13:42.326ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
170052025-04-30T06:13:42.326ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
170062025-04-30T06:13:42.326ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170072025-04-30T06:13:42.326ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170082025-04-30T06:13:42.327ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170092025-04-30T06:13:42.329ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170102025-04-30T06:13:42.331ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
170112025-04-30T06:13:42.334ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
170122025-04-30T06:13:42.334ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170132025-04-30T06:13:42.334ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170142025-04-30T06:13:42.335ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170152025-04-30T06:13:42.383ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
170162025-04-30T06:13:42.383ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
170172025-04-30T06:13:42.383ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
170182025-04-30T06:13:42.383ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
170192025-04-30T06:13:42.384ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
170202025-04-30T06:13:42.402ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
170212025-04-30T06:13:42.402ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
170222025-04-30T06:13:42.403ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:46499 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
170232025-04-30T06:13:42.403ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:46499 req_id = c98b8196-ec57-4625-877b-a61e0dbb833f sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
170242025-04-30T06:13:42.405ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170252025-04-30T06:13:42.405ZINFOomicron-dev (dropshot): request completed latency_us = 2072 local_addr = [::1]:61881 method = GET remote_addr = [::1]:46499 req_id = c98b8196-ec57-4625-877b-a61e0dbb833f response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
170262025-04-30T06:13:42.406ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "c98b8196-ec57-4625-877b-a61e0dbb833f", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
170272025-04-30T06:13:42.414ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
170282025-04-30T06:13:42.414ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
170292025-04-30T06:13:42.414ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
170302025-04-30T06:13:42.414ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
170312025-04-30T06:13:42.414ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
170322025-04-30T06:13:42.414ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170332025-04-30T06:13:42.414ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170342025-04-30T06:13:42.414ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170352025-04-30T06:13:42.419ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
170362025-04-30T06:13:42.419ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
170372025-04-30T06:13:42.419ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
170382025-04-30T06:13:42.423ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
170392025-04-30T06:13:42.424ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170402025-04-30T06:13:42.430ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
170412025-04-30T06:13:42.430ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170422025-04-30T06:13:42.431ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170432025-04-30T06:13:42.433ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170442025-04-30T06:13:42.437ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
170452025-04-30T06:13:42.437ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
170462025-04-30T06:13:42.445ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
170472025-04-30T06:13:42.507ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
170482025-04-30T06:13:42.538ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
170492025-04-30T06:13:42.539ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
170502025-04-30T06:13:42.540ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 477.042305ms iteration = 64 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
170512025-04-30T06:13:42.541ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 65 reason = Signaled
170522025-04-30T06:13:42.541ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
170532025-04-30T06:13:42.541ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
170542025-04-30T06:13:42.541ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170552025-04-30T06:13:42.541ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170562025-04-30T06:13:42.558ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170572025-04-30T06:13:42.559ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
170582025-04-30T06:13:42.559ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
170592025-04-30T06:13:42.559ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
170602025-04-30T06:13:42.559ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
170612025-04-30T06:13:42.561ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
170622025-04-30T06:13:42.578ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170632025-04-30T06:13:42.584ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
170642025-04-30T06:13:42.584ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
170652025-04-30T06:13:42.584ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
170662025-04-30T06:13:42.584ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
170672025-04-30T06:13:42.585ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
170682025-04-30T06:13:42.592ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
170692025-04-30T06:13:42.592ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
170702025-04-30T06:13:42.592ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
170712025-04-30T06:13:42.593ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
170722025-04-30T06:13:42.598ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
170732025-04-30T06:13:42.598ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
170742025-04-30T06:13:42.601ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
170752025-04-30T06:13:42.601ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
170762025-04-30T06:13:42.602ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: 43 49 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 CI..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:64514 req_id = b00113dc-a8ad-4637-9a30-9235877157c6
170772025-04-30T06:13:42.602ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 17225,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:64514 req_id = b00113dc-a8ad-4637-9a30-9235877157c6
170782025-04-30T06:13:42.602ZDEBGomicron-dev (store): query key key = _mgs._tcp
170792025-04-30T06:13:42.602ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
170802025-04-30T06:13:42.602ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:64514 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = b00113dc-a8ad-4637-9a30-9235877157c6
170812025-04-30T06:13:42.602ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
170822025-04-30T06:13:42.603ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2487, tv_nsec: 401732576 } })
170832025-04-30T06:13:42.603ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
170842025-04-30T06:13:42.603ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 82 e8 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ............-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:54138 req_id = 9e63d4dc-8dfe-41de-9d12-e10acc86a3a9
170852025-04-30T06:13:42.603ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 33512,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:54138 req_id = 9e63d4dc-8dfe-41de-9d12-e10acc86a3a9
170862025-04-30T06:13:42.603ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
170872025-04-30T06:13:42.603ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:54138 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 9e63d4dc-8dfe-41de-9d12-e10acc86a3a9
170882025-04-30T06:13:42.648ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
170892025-04-30T06:13:42.648ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: fa 6a 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c .j..........._cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:65220 req_id = a28d1cff-0855-49d5-9cd3-e7d46c8bfec8
170902025-04-30T06:13:42.649ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64106,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:65220 req_id = a28d1cff-0855-49d5-9cd3-e7d46c8bfec8
170912025-04-30T06:13:42.649ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
170922025-04-30T06:13:42.649ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 64106, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:65220 req_id = a28d1cff-0855-49d5-9cd3-e7d46c8bfec8
170932025-04-30T06:13:42.649ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
170942025-04-30T06:13:42.649ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
170952025-04-30T06:13:42.649ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
170962025-04-30T06:13:42.649ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "5f241c53-1d94-4034-8f98-6845fae286b1", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
170972025-04-30T06:13:42.649ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
170982025-04-30T06:13:42.649ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d6b2179-ae05-452a-8183-6b17e87c076d", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
170992025-04-30T06:13:42.649ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
171002025-04-30T06:13:42.651ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd1a50d4-4176-4b0b-82a3-b2f2a60998b9", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171012025-04-30T06:13:42.651ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
171022025-04-30T06:13:42.653ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "bb9f12e5-55d8-4002-b8cf-94221c3ac010", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171032025-04-30T06:13:42.653ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
171042025-04-30T06:13:42.655ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "514c0326-a05c-4f34-84e5-28662c64a470", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171052025-04-30T06:13:42.655ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
171062025-04-30T06:13:42.655ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
171072025-04-30T06:13:42.658ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "4d974938-9dc3-432c-b909-241e1a128061", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171082025-04-30T06:13:42.659ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
171092025-04-30T06:13:42.661ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "03573649-e93b-41c0-be13-959586b4a550", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171102025-04-30T06:13:42.661ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
171112025-04-30T06:13:42.663ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "67b0dba7-3fc6-4aea-ac61-e0e7369eb72b", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171122025-04-30T06:13:42.663ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
171132025-04-30T06:13:42.664ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c956b5f-5cb1-4b4d-b1b5-df750eee3f9d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171142025-04-30T06:13:42.664ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
171152025-04-30T06:13:42.665ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb276989-b867-488c-ba57-0387a74b5ce0", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171162025-04-30T06:13:42.666ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
171172025-04-30T06:13:42.667ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9b745092-6fd8-424a-b3cb-82ef383fe073", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171182025-04-30T06:13:42.667ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
171192025-04-30T06:13:42.668ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5012d1e8-91bd-49b7-b826-0d0ddbefdcda", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171202025-04-30T06:13:42.668ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
171212025-04-30T06:13:42.670ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e27f5406-2243-4d49-a3a7-b0205873208b", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171222025-04-30T06:13:42.670ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
171232025-04-30T06:13:42.672ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e171da8-dcd0-401f-8944-19e373f760fa", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171242025-04-30T06:13:42.672ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
171252025-04-30T06:13:42.674ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e222c73-d8c2-4c6d-9b7b-83786328d10e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171262025-04-30T06:13:42.675ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
171272025-04-30T06:13:42.677ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b13862c4-3b9f-47a3-903f-05ff52c8e121", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171282025-04-30T06:13:42.677ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
171292025-04-30T06:13:42.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
171302025-04-30T06:13:42.680ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
171312025-04-30T06:13:42.681ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "8439b542-7ccd-49e8-ae62-ce957d583657", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171322025-04-30T06:13:42.681ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
171332025-04-30T06:13:42.681ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a83273b-0757-4016-a283-96328139212e uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
171342025-04-30T06:13:42.682ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a83273b-0757-4016-a283-96328139212e resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
171352025-04-30T06:13:42.682ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a83273b-0757-4016-a283-96328139212e roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
171362025-04-30T06:13:42.683ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a83273b-0757-4016-a283-96328139212e resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
171372025-04-30T06:13:42.684ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "653134b7-bf52-44ca-91b5-e6fef5796197", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171382025-04-30T06:13:42.684ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
171392025-04-30T06:13:42.691ZINFOomicron-dev (dropshot_internal): request completed latency_us = 9824 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 8a83273b-0757-4016-a283-96328139212e response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
171402025-04-30T06:13:42.692ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "8a83273b-0757-4016-a283-96328139212e", "content-length": "401", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171412025-04-30T06:13:42.693ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
171422025-04-30T06:13:42.693ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 51772906-2988-4d74-9e90-3df079d2df34 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
171432025-04-30T06:13:42.694ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 51772906-2988-4d74-9e90-3df079d2df34 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
171442025-04-30T06:13:42.694ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 51772906-2988-4d74-9e90-3df079d2df34 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
171452025-04-30T06:13:42.695ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 51772906-2988-4d74-9e90-3df079d2df34 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
171462025-04-30T06:13:42.696ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "a4b22aaa-86a3-4483-ba14-d66f31e0188e", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171472025-04-30T06:13:42.697ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
171482025-04-30T06:13:42.700ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "caab6df7-544f-4ff8-bd08-414db99bb507", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171492025-04-30T06:13:42.700ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
171502025-04-30T06:13:42.701ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "d26de38d-0993-4a63-9a76-0cc611dd9e6d", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171512025-04-30T06:13:42.701ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
171522025-04-30T06:13:42.703ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd245d0f-16ae-4a00-b1bc-ef06060776df", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171532025-04-30T06:13:42.703ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
171542025-04-30T06:13:42.704ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c55d1328-1216-45a8-bd49-54e8273f78c7", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171552025-04-30T06:13:42.705ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
171562025-04-30T06:13:42.706ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
171572025-04-30T06:13:42.706ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
171582025-04-30T06:13:42.706ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
171592025-04-30T06:13:42.706ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
171602025-04-30T06:13:42.706ZINFOomicron-dev (dropshot_internal): request completed latency_us = 12759 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 51772906-2988-4d74-9e90-3df079d2df34 response_code = 200 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D
171612025-04-30T06:13:42.707ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
171622025-04-30T06:13:42.707ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlNmJmZjFmZi0yNGZiLTQ5ZGMtYTU0ZS1jNmEzNTBjZDRkNmMifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "51772906-2988-4d74-9e90-3df079d2df34", "content-length": "29", "date": "Wed, 30 Apr 2025 06:13:41 GMT"} })
171632025-04-30T06:13:42.707ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "384092fc-e5eb-49d8-adf5-e503cb3ceeb0", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171642025-04-30T06:13:42.707ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
171652025-04-30T06:13:42.707ZTRAComicron-dev (oximeter-agent): ignoring request to update existing metric producer, since the endpoint information is the same as the existing collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
171662025-04-30T06:13:42.707ZINFOomicron-dev (oximeter-agent): refreshed list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:637 n_current_tasks = 2 n_pruned_tasks = 0
171672025-04-30T06:13:42.708ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
171682025-04-30T06:13:42.710ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
171692025-04-30T06:13:42.710ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6d486e6d-2792-4bef-9801-ad78814fc672", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171702025-04-30T06:13:42.711ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
171712025-04-30T06:13:42.713ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "c6fbaf6b-cbdc-471e-babd-4c27af646553", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171722025-04-30T06:13:42.713ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
171732025-04-30T06:13:42.715ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
171742025-04-30T06:13:42.715ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "b5b1f002-fc0b-4f0e-8afb-25db81389636", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171752025-04-30T06:13:42.715ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
171762025-04-30T06:13:42.715ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
171772025-04-30T06:13:42.715ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
171782025-04-30T06:13:42.716ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
171792025-04-30T06:13:42.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
171802025-04-30T06:13:42.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
171812025-04-30T06:13:42.720ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2bd68c04-a000-4fdb-8efa-4b7bdd30237d", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171822025-04-30T06:13:42.720ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
171832025-04-30T06:13:42.721ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
171842025-04-30T06:13:42.722ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
171852025-04-30T06:13:42.722ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
171862025-04-30T06:13:42.724ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4081bcd3-9393-42a4-9402-535e0a4c73a7", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171872025-04-30T06:13:42.725ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
171882025-04-30T06:13:42.727ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "44fda8f6-8d01-4c8e-9b8d-2d1c3f60c22e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171892025-04-30T06:13:42.727ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
171902025-04-30T06:13:42.727ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "418acea1-6e33-4212-9894-5f7840ce651d", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171912025-04-30T06:13:42.728ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
171922025-04-30T06:13:42.729ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "0e1002ec-7bb5-4b62-98b7-03d3d7232c9e", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171932025-04-30T06:13:42.729ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
171942025-04-30T06:13:42.730ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cc8d1d03-44b4-48f9-85cb-76bdb2b0086e", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171952025-04-30T06:13:42.730ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
171962025-04-30T06:13:42.731ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "e2bf0e3e-7e3c-440d-b282-e2c139b3d09a", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171972025-04-30T06:13:42.731ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
171982025-04-30T06:13:42.732ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f9bb466-3d7c-4751-b089-95795c3e94f7", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
171992025-04-30T06:13:42.732ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
172002025-04-30T06:13:42.734ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f342596d-9dba-403b-98a9-90a4633fbeb3", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172012025-04-30T06:13:42.734ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
172022025-04-30T06:13:42.736ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
172032025-04-30T06:13:42.736ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "7a0da928-c063-4737-81e8-979678c20b57", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172042025-04-30T06:13:42.736ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
172052025-04-30T06:13:42.738ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "e23163e6-d5da-454a-b437-943883d93c31", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172062025-04-30T06:13:42.738ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
172072025-04-30T06:13:42.738ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
172082025-04-30T06:13:42.738ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
172092025-04-30T06:13:42.738ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
172102025-04-30T06:13:42.751ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b4bbafb4-12e7-4235-80e0-0842b3717885", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172112025-04-30T06:13:42.751ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
172122025-04-30T06:13:42.754ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a08deb8-4e7a-4be1-8328-71400ec90005", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172132025-04-30T06:13:42.755ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
172142025-04-30T06:13:42.762ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "ab12e263-91b3-46a2-b775-0865e47d54b5", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172152025-04-30T06:13:42.762ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
172162025-04-30T06:13:42.762ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "63192dec-33e9-4c96-9ce5-1689687fc444", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172172025-04-30T06:13:42.762ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
172182025-04-30T06:13:42.763ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "cddac16b-be0d-4cb6-a018-630a81e5bbe8", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172192025-04-30T06:13:42.763ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
172202025-04-30T06:13:42.763ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
172212025-04-30T06:13:42.765ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "95fd832c-3fc4-4027-a582-6424a8034c99", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172222025-04-30T06:13:42.765ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
172232025-04-30T06:13:42.766ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "c14c4ed7-0737-4fa8-af9a-191d0ad7674c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172242025-04-30T06:13:42.767ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
172252025-04-30T06:13:42.767ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
172262025-04-30T06:13:42.767ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
172272025-04-30T06:13:42.767ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172282025-04-30T06:13:42.768ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
172292025-04-30T06:13:42.772ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172302025-04-30T06:13:42.782ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
172312025-04-30T06:13:42.783ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
172322025-04-30T06:13:42.783ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
172332025-04-30T06:13:42.783ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
172342025-04-30T06:13:42.784ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172352025-04-30T06:13:42.785ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
172362025-04-30T06:13:42.814ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172372025-04-30T06:13:42.819ZTRAComicron-dev (InternalApi): authorize begin action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory
172382025-04-30T06:13:42.819ZTRAComicron-dev (InternalApi): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
172392025-04-30T06:13:42.819ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
172402025-04-30T06:13:42.819ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
172412025-04-30T06:13:42.820ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
172422025-04-30T06:13:42.824ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
172432025-04-30T06:13:42.824ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
172442025-04-30T06:13:42.824ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172452025-04-30T06:13:42.825ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
172462025-04-30T06:13:42.829ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172472025-04-30T06:13:42.834ZDEBGomicron-dev (InternalApi): authorize result action = Read actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Inventory result = Ok(())
172482025-04-30T06:13:42.834ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
172492025-04-30T06:13:42.834ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
172502025-04-30T06:13:42.848ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
172512025-04-30T06:13:42.891ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
172522025-04-30T06:13:42.891ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
172532025-04-30T06:13:42.892ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:58063 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
172542025-04-30T06:13:42.893ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:58063 req_id = 7cfadd31-9eb2-4508-a69e-144e4d90c8e8 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
172552025-04-30T06:13:42.894ZINFOomicron-dev (dropshot): request completed latency_us = 477 local_addr = [::1]:61881 method = GET remote_addr = [::1]:58063 req_id = 7cfadd31-9eb2-4508-a69e-144e4d90c8e8 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
172562025-04-30T06:13:42.894ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "7cfadd31-9eb2-4508-a69e-144e4d90c8e8", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
172572025-04-30T06:13:42.898ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
172582025-04-30T06:13:42.898ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
172592025-04-30T06:13:42.898ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
172602025-04-30T06:13:42.898ZTRAComicron-dev (InternalApi): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database
172612025-04-30T06:13:42.898ZDEBGomicron-dev (InternalApi): roles roles = RoleSet { roles: {} }
172622025-04-30T06:13:42.899ZDEBGomicron-dev (InternalApi): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) resource = Database result = Ok(())
172632025-04-30T06:13:42.901ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
172642025-04-30T06:13:42.901ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
172652025-04-30T06:13:42.901ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
172662025-04-30T06:13:42.901ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172672025-04-30T06:13:42.902ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
172682025-04-30T06:13:42.905ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172692025-04-30T06:13:42.913ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
172702025-04-30T06:13:42.915ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
172712025-04-30T06:13:42.915ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172722025-04-30T06:13:42.917ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
172732025-04-30T06:13:42.966ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
172742025-04-30T06:13:42.967ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
172752025-04-30T06:13:42.967ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 427.133335ms iteration = 65 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
172762025-04-30T06:13:42.967ZDEBGomicron-dev (BackgroundTasks): activating background_task = inventory_collection iteration = 66 reason = Signaled
172772025-04-30T06:13:42.967ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
172782025-04-30T06:13:42.967ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
172792025-04-30T06:13:42.967ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
172802025-04-30T06:13:42.968ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172812025-04-30T06:13:42.969ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
172822025-04-30T06:13:42.972ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
172832025-04-30T06:13:42.981ZDEBGomicron-dev: attempt to fetch latest inventory collection result = Ok("found")
172842025-04-30T06:13:42.985ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
172852025-04-30T06:13:42.985ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
172862025-04-30T06:13:42.985ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
172872025-04-30T06:13:42.986ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
172882025-04-30T06:13:42.991ZDEBGomicron-dev (BackgroundTasks): inventory_prune_one: nothing eligible for removal (too few) background_task = inventory_collection candidates = [(56c2f67f-67f1-43f2-8f3d-f724f719303e, 0)]
172892025-04-30T06:13:42.992ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal
172902025-04-30T06:13:42.993ZTRAComicron-dev (dns): buffer buffer = Length: 67 (0x43) bytes\n0000: c3 69 01 00 00 01 00 00 00 00 00 01 04 5f 6d 67 .i..........._mg\n0010: 73 04 5f 74 63 70 0d 63 6f 6e 74 72 6f 6c 2d 70 s._tcp.control-p\n0020: 6c 61 6e 65 05 6f 78 69 64 65 08 69 6e 74 65 72 lane.oxide.inter\n0030: 6e 61 6c 00 00 21 00 01 00 00 29 04 d0 00 00 00 nal..!....).....\n0040: 00 00 00 ... kind = dns peer_addr = [::1]:63420 req_id = f47ea7d0-95f1-4a28-be3b-23073a848691
172912025-04-30T06:13:42.993ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 50025,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_mgs._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_mgs._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 4,\n 95,\n 109,\n 103,\n 115,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:63420 req_id = f47ea7d0-95f1-4a28-be3b-23073a848691
172922025-04-30T06:13:42.995ZDEBGomicron-dev (store): query key key = _mgs._tcp
172932025-04-30T06:13:42.998ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
172942025-04-30T06:13:42.998ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:63420 query = LowerQuery { name: LowerName(Name("_mgs._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_mgs._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = f47ea7d0-95f1-4a28-be3b-23073a848691
172952025-04-30T06:13:42.999ZDEBGomicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _mgs._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_mgs._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_mgs._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 40478, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2487, tv_nsec: 798358507 } })
172962025-04-30T06:13:42.999ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
172972025-04-30T06:13:43.000ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: ca 30 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e .0..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60789 req_id = 3bac6df6-392f-4161-be82-78240f7bda8a
172982025-04-30T06:13:43.001ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 51760,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60789 req_id = 3bac6df6-392f-4161-be82-78240f7bda8a
172992025-04-30T06:13:43.001ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
173002025-04-30T06:13:43.001ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60789 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = 3bac6df6-392f-4161-be82-78240f7bda8a
173012025-04-30T06:13:43.050ZTRAComicron-dev (DnsResolver): lookup_all_socket_v6 srv dns_name = _clickhouse-admin-keeper._tcp.control-plane.oxide.internal
173022025-04-30T06:13:43.050ZTRAComicron-dev (dns): buffer buffer = Length: 87 (0x57) bytes\n0000: 58 1a 01 00 00 01 00 00 00 00 00 01 18 5f 63 6c X............_cl\n0010: 69 63 6b 68 6f 75 73 65 2d 61 64 6d 69 6e 2d 6b ickhouse-admin-k\n0020: 65 65 70 65 72 04 5f 74 63 70 0d 63 6f 6e 74 72 eeper._tcp.contr\n0030: 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 69 ol-plane.oxide.i\n0040: 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 04 nternal..!....).\n0050: d0 00 00 00 00 00 00 ....... kind = dns peer_addr = [::1]:56409 req_id = 1527db25-8626-40e9-97ab-c491eb0b732d
173032025-04-30T06:13:43.050ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 22554,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 24,\n 95,\n 99,\n 108,\n 105,\n 99,\n 107,\n 104,\n 111,\n 117,\n 115,\n 101,\n 45,\n 97,\n 100,\n 109,\n 105,\n 110,\n 45,\n 107,\n 101,\n 101,\n 112,\n 101,\n 114,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:56409 req_id = 1527db25-8626-40e9-97ab-c491eb0b732d
173042025-04-30T06:13:43.051ZDEBGomicron-dev (store): query key key = _clickhouse-admin-keeper._tcp
173052025-04-30T06:13:43.051ZERROomicron-dev (dns): failed to handle incoming DNS message: MessageRequest { header: Header { id: 22554, message_type: Query, op_code: Query, authoritative: false, truncation: false, recursion_desired: true, recursion_available: false, authentic_data: false, checking_disabled: false, response_code: NoError, query_count: 1, answer_count: 0, name_server_count: 0, additional_count: 1, }, query: WireQuery { query: LowerQuery { name: LowerName( Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), ), original: Query { name: Name("_clickhouse-admin-keeper._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN, }, }, original: [ 24, 95, 99, 108, 105, 99, 107, 104, 111, 117, 115, 101, 45, 97, 100, 109, 105, 110, 45, 107, 101, 101, 112, 101, 114, 4, 95, 116, 99, 112, 13, 99, 111, 110, 116, 114, 111, 108, 45, 112, 108, 97, 110, 101, 5, 111, 120, 105, 100, 101, 8, 105, 110, 116, 101, 114, 110, 97, 108, 0, 0, 33, 0, 1, ], }, answers: [], name_servers: [], additionals: [], sig0: [], edns: Some( Edns { rcode_high: 0, version: 0, dnssec_ok: false, max_payload: 1232, options: OPT { options: {}, }, }, ), } NXDOMAIN: no records found for name: "_clickhouse-admin-keeper._tcp.control-plane.oxide.internal." kind = dns peer_addr = [::1]:56409 req_id = 1527db25-8626-40e9-97ab-c491eb0b732d
173062025-04-30T06:13:43.051ZDEBGomicron-dev (BackgroundTasks): begin collection background_task = inventory_collection
173072025-04-30T06:13:43.051ZDEBGomicron-dev (BackgroundTasks): begin collection from MGS background_task = inventory_collection mgs_url = http://[::1]:40478
173082025-04-30T06:13:43.051ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/ignition
173092025-04-30T06:13:43.053ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/ignition", status: 200, headers: {"content-type": "application/json", "x-request-id": "a0377171-05c5-4868-a5fc-51482ef85bea", "content-length": "835", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173102025-04-30T06:13:43.053ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0
173112025-04-30T06:13:43.057ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb9881cc-c192-4323-829b-cc3803ccb31a", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173122025-04-30T06:13:43.057ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0
173132025-04-30T06:13:43.061ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "412f16bc-738f-4675-baaa-9a3e40306a65", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173142025-04-30T06:13:43.062ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1
173152025-04-30T06:13:43.065ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d435add2-e44c-4691-9635-574241e09a84", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173162025-04-30T06:13:43.065ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0
173172025-04-30T06:13:43.070ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "9228313d-8b2a-453b-9da6-e9981677a271", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173182025-04-30T06:13:43.070ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1
173192025-04-30T06:13:43.073ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b67bcc92-7357-458c-b5fc-6649e43a781a", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173202025-04-30T06:13:43.073ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0
173212025-04-30T06:13:43.075ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c39f3118-cf93-446a-a8a7-238891a34d81", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173222025-04-30T06:13:43.075ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1
173232025-04-30T06:13:43.078ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a2801bd-5318-4c31-930b-9c59728b38bb", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173242025-04-30T06:13:43.078ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cmpa
173252025-04-30T06:13:43.079ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "3862dddb-dedd-42a0-885e-fd39218d6ad7", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173262025-04-30T06:13:43.079ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
173272025-04-30T06:13:43.081ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "02dedcaa-7e67-4feb-9278-8185d07db25c", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173282025-04-30T06:13:43.081ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
173292025-04-30T06:13:43.082ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "9a23b126-a7dd-4068-86f3-f5aee422c29a", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173302025-04-30T06:13:43.082ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/0/component/rot/cfpa
173312025-04-30T06:13:43.084ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "704182c1-f77e-44dd-aea0-60f8671e0831", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173322025-04-30T06:13:43.084ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1
173332025-04-30T06:13:43.086ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "b72b18dc-2868-4e77-aa30-13d04b5ffe07", "content-length": "734", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173342025-04-30T06:13:43.087ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0
173352025-04-30T06:13:43.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
173362025-04-30T06:13:43.089ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
173372025-04-30T06:13:43.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
173382025-04-30T06:13:43.090ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
173392025-04-30T06:13:43.090ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "6a538c95-995d-468b-a9d6-66e52c464c76", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173402025-04-30T06:13:43.090ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1
173412025-04-30T06:13:43.095ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "96a2c3f1-3c13-449a-8f1e-a6a89464d52e", "content-length": "111", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173422025-04-30T06:13:43.095ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0
173432025-04-30T06:13:43.096ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
173442025-04-30T06:13:43.096ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
173452025-04-30T06:13:43.098ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
173462025-04-30T06:13:43.098ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
173472025-04-30T06:13:43.101ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "4283611c-5ce0-4174-a939-10016c57d671", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173482025-04-30T06:13:43.101ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1
173492025-04-30T06:13:43.110ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2e541917-341c-4b10-a751-84643bd3bb8e", "content-length": "170", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173502025-04-30T06:13:43.113ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0
173512025-04-30T06:13:43.120ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "3d896367-38b9-47bf-b3b2-a4aaa821f14c", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173522025-04-30T06:13:43.120ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1
173532025-04-30T06:13:43.122ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "404dd311-ab9a-4657-a6e4-e028be17ac47", "content-length": "179", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173542025-04-30T06:13:43.122ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cmpa
173552025-04-30T06:13:43.124ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "fd3637b8-7dfc-4541-b18e-4e0a07a674a5", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173562025-04-30T06:13:43.124ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
173572025-04-30T06:13:43.126ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "6aa31806-cfed-4403-b0c2-7e81ccb7939a", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173582025-04-30T06:13:43.126ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
173592025-04-30T06:13:43.127ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "7326e2b2-8427-416f-9448-3517300a15a5", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173602025-04-30T06:13:43.128ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/switch/1/component/rot/cfpa
173612025-04-30T06:13:43.130ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/switch/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "325e30ba-53c2-41eb-889c-a94f09ea8d8c", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173622025-04-30T06:13:43.130ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0
173632025-04-30T06:13:43.132ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0", status: 200, headers: {"content-type": "application/json", "x-request-id": "55e4689d-0421-4092-bde2-e73ab0231904", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173642025-04-30T06:13:43.132ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0
173652025-04-30T06:13:43.134ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "f9701a0c-4ce9-433d-a638-c799a5459f52", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173662025-04-30T06:13:43.135ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1
173672025-04-30T06:13:43.137ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "0022c108-0f15-4fd9-ae7d-96433d3bf99a", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173682025-04-30T06:13:43.137ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0
173692025-04-30T06:13:43.139ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "c7d0d7c3-50c4-438a-b779-e240ba8430e7", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173702025-04-30T06:13:43.140ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1
173712025-04-30T06:13:43.144ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "3127b1dc-6d7c-4b01-8470-7705c59e5872", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173722025-04-30T06:13:43.144ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0
173732025-04-30T06:13:43.151ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "93687d04-72ba-469a-bc5d-a79f5880b28e", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173742025-04-30T06:13:43.151ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1
173752025-04-30T06:13:43.153ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "f0b6fbaa-a52d-4913-9fa9-837afa48d750", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173762025-04-30T06:13:43.153ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cmpa
173772025-04-30T06:13:43.154ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "b851cba6-6e6c-478e-bc8c-4e2ee00870e0", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173782025-04-30T06:13:43.154ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
173792025-04-30T06:13:43.156ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "86572a25-32aa-4396-a890-b40d3421afcc", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
173802025-04-30T06:13:43.156ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
173812025-04-30T06:13:43.158ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "a08f4f9c-28e3-4977-a4b9-26f76ecb240d", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173822025-04-30T06:13:43.159ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/0/component/rot/cfpa
173832025-04-30T06:13:43.160ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/0/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "5b6e1917-3cd8-4209-bd2a-c9b444e378bd", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173842025-04-30T06:13:43.161ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1
173852025-04-30T06:13:43.162ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e38fe981-6e85-4250-ad81-5cb6e1b6ec01", "content-length": "723", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173862025-04-30T06:13:43.162ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0
173872025-04-30T06:13:43.164ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "8134ebbc-4c89-460c-83c9-05647f5d6868", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173882025-04-30T06:13:43.164ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1
173892025-04-30T06:13:43.167ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/sp/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "e22509a4-3f6c-4d27-b076-ab9981b508b7", "content-length": "109", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173902025-04-30T06:13:43.168ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0
173912025-04-30T06:13:43.171ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "13136b94-ebe0-40f9-b56c-90c5c723ee71", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173922025-04-30T06:13:43.171ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1
173932025-04-30T06:13:43.175ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "d8db57b0-c651-42d1-a3aa-f664dd919ed3", "content-length": "169", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173942025-04-30T06:13:43.176ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0
173952025-04-30T06:13:43.178ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=0", status: 200, headers: {"content-type": "application/json", "x-request-id": "2a5476c5-0c36-4c45-bfa2-ed00e5429889", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173962025-04-30T06:13:43.178ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1
173972025-04-30T06:13:43.180ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/stage0/caboose?firmware_slot=1", status: 200, headers: {"content-type": "application/json", "x-request-id": "2f833ed4-b002-45b0-8e01-723bd81c467c", "content-length": "178", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
173982025-04-30T06:13:43.180ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = None gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cmpa
173992025-04-30T06:13:43.181ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cmpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "78e76bc0-88f8-44a7-b02b-d72b5f1963aa", "content-length": "702", "date": "Wed, 30 Apr 2025 06:13:42 GMT"} })
174002025-04-30T06:13:43.181ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
174012025-04-30T06:13:43.182ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "ee21b73c-16ec-49d3-b88f-3568a84f04a1", "content-length": "727", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
174022025-04-30T06:13:43.182ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
174032025-04-30T06:13:43.183ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "f1fe4547-b42e-4453-9656-15566b91b9b7", "content-length": "729", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
174042025-04-30T06:13:43.183ZDEBGomicron-dev (BackgroundTasks): client request background_task = inventory_collection body = Some(Body) gateway_url = http://[::1]:40478 method = GET uri = http://[::1]:40478/sp/sled/1/component/rot/cfpa
174052025-04-30T06:13:43.184ZDEBGomicron-dev (BackgroundTasks): client response background_task = inventory_collection gateway_url = http://[::1]:40478 result = Ok(Response { url: "http://[::1]:40478/sp/sled/1/component/rot/cfpa", status: 200, headers: {"content-type": "application/json", "x-request-id": "1b826799-8f6a-4a44-a517-b2cd810a34b0", "content-length": "728", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
174062025-04-30T06:13:43.185ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
174072025-04-30T06:13:43.185ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
174082025-04-30T06:13:43.185ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
174092025-04-30T06:13:43.185ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
174102025-04-30T06:13:43.185ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
174112025-04-30T06:13:43.190ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174122025-04-30T06:13:43.191ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
174132025-04-30T06:13:43.200ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
174142025-04-30T06:13:43.200ZTRAComicron-dev (BackgroundTasks): authorize begin action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet
174152025-04-30T06:13:43.200ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
174162025-04-30T06:13:43.200ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
174172025-04-30T06:13:43.200ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
174182025-04-30T06:13:43.200ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
174192025-04-30T06:13:43.204ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174202025-04-30T06:13:43.209ZDEBGomicron-dev (BackgroundTasks): authorize result action = ListChildren actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Fleet result = Ok(())
174212025-04-30T06:13:43.210ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
174222025-04-30T06:13:43.210ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
174232025-04-30T06:13:43.211ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
174242025-04-30T06:13:43.268ZDEBGomicron-dev (BackgroundTasks): begin collection from Sled Agent background_task = inventory_collection sled_agent_url = http://[::1]:61881
174252025-04-30T06:13:43.268ZDEBGomicron-dev (BackgroundTasks): client request SledAgent = http://[::1]:61881 background_task = inventory_collection body = None method = GET uri = http://[::1]:61881/inventory
174262025-04-30T06:13:43.269ZINFOomicron-dev (dropshot): accepted connection local_addr = [::1]:61881 remote_addr = [::1]:60139 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
174272025-04-30T06:13:43.270ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:61881 method = GET remote_addr = [::1]:60139 req_id = f4879a89-2e8c-43c5-a5d4-981d6c6a0555 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
174282025-04-30T06:13:43.270ZINFOomicron-dev (dropshot): request completed latency_us = 569 local_addr = [::1]:61881 method = GET remote_addr = [::1]:60139 req_id = f4879a89-2e8c-43c5-a5d4-981d6c6a0555 response_code = 200 sled_id = b6d65341-167c-41df-9b5c-41cded99c229 uri = /inventory
174292025-04-30T06:13:43.271ZDEBGomicron-dev (BackgroundTasks): client response SledAgent = http://[::1]:61881 background_task = inventory_collection result = Ok(Response { url: "http://[::1]:61881/inventory", status: 200, headers: {"content-type": "application/json", "x-request-id": "f4879a89-2e8c-43c5-a5d4-981d6c6a0555", "content-length": "3128", "date": "Wed, 30 Apr 2025 06:13:43 GMT"} })
174302025-04-30T06:13:43.274ZDEBGomicron-dev (BackgroundTasks): begin collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
174312025-04-30T06:13:43.274ZDEBGomicron-dev (BackgroundTasks): end collecting all keepers background_task = inventory_collection nkeeper_admin_clients = 0
174322025-04-30T06:13:43.274ZDEBGomicron-dev (BackgroundTasks): finished collection background_task = inventory_collection
174332025-04-30T06:13:43.276ZTRAComicron-dev (BackgroundTasks): authorize begin action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory
174342025-04-30T06:13:43.276ZTRAComicron-dev (BackgroundTasks): loading roles actor = Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. } background_task = inventory_collection resource_id = 001de000-1334-4000-8000-000000000000 resource_type = Fleet
174352025-04-30T06:13:43.277ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
174362025-04-30T06:13:43.277ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
174372025-04-30T06:13:43.277ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
174382025-04-30T06:13:43.281ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {(Fleet, 001de000-1334-4000-8000-000000000000, "admin")} }
174392025-04-30T06:13:43.287ZDEBGomicron-dev (BackgroundTasks): authorize result action = Modify actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Inventory result = Ok(())
174402025-04-30T06:13:43.287ZTRAComicron-dev (BackgroundTasks): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database
174412025-04-30T06:13:43.287ZDEBGomicron-dev (BackgroundTasks): roles background_task = inventory_collection roles = RoleSet { roles: {} }
174422025-04-30T06:13:43.288ZDEBGomicron-dev (BackgroundTasks): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) background_task = inventory_collection resource = Database result = Ok(())
174432025-04-30T06:13:43.319ZWARNomicron-dev (datastore): Non-retryable transaction failure err = DatabaseError(UniqueViolation, "duplicate key value violates unique constraint \\"inv_caboose_pkey\\"") transaction = inventory_insert_collection
174442025-04-30T06:13:43.719ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
174452025-04-30T06:13:43.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
174462025-04-30T06:13:43.721ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
174472025-04-30T06:13:43.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
174482025-04-30T06:13:43.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
174492025-04-30T06:13:43.722ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
174502025-04-30T06:13:43.794ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
174512025-04-30T06:13:43.800ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 4e 18 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 N............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:51955 req_id = 5a61f1aa-5165-463f-923a-522c8ec15a64
174522025-04-30T06:13:43.800ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 19992,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51955 req_id = 5a61f1aa-5165-463f-923a-522c8ec15a64
174532025-04-30T06:13:43.800ZDEBGomicron-dev (store): query key key = _dendrite._tcp
174542025-04-30T06:13:43.800ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
174552025-04-30T06:13:43.800ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51955 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 5a61f1aa-5165-463f-923a-522c8ec15a64
174562025-04-30T06:13:43.801ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2488, tv_nsec: 600063752 } })
174572025-04-30T06:13:43.801ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
174582025-04-30T06:13:43.801ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 5f bf 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e _...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:57971 req_id = a729e9bd-c8dd-4333-99ea-6343f0a91ea1
174592025-04-30T06:13:43.802ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 24511,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:57971 req_id = a729e9bd-c8dd-4333-99ea-6343f0a91ea1
174602025-04-30T06:13:43.802ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
174612025-04-30T06:13:43.802ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:57971 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = a729e9bd-c8dd-4333-99ea-6343f0a91ea1
174622025-04-30T06:13:43.806ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = bfd_manager
174632025-04-30T06:13:43.870ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = bfd_manager zone_address = ::1
174642025-04-30T06:13:43.870ZDEBGomicron-dev (MgsClient): client request background_task = bfd_manager body = None method = GET uri = http://[::1]:12225/local/switch-id
174652025-04-30T06:13:43.873ZDEBGomicron-dev (MgsClient): client response background_task = bfd_manager result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
174662025-04-30T06:13:43.873ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = bfd_manager reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
174672025-04-30T06:13:43.876ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = bfd_manager
174682025-04-30T06:13:43.876ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
174692025-04-30T06:13:43.877ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: 73 e0 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 s............_de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:51136 req_id = 617f8cfc-7977-4e37-a9cc-636fce4a0f6e
174702025-04-30T06:13:43.877ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 29664,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:51136 req_id = 617f8cfc-7977-4e37-a9cc-636fce4a0f6e
174712025-04-30T06:13:43.877ZDEBGomicron-dev (store): query key key = _dendrite._tcp
174722025-04-30T06:13:43.877ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
174732025-04-30T06:13:43.877ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:51136 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 617f8cfc-7977-4e37-a9cc-636fce4a0f6e
174742025-04-30T06:13:43.878ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2488, tv_nsec: 677034782 } })
174752025-04-30T06:13:43.878ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
174762025-04-30T06:13:43.878ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 5a 59 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e ZY..........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:58020 req_id = dc72671c-44f5-484d-a535-a17d866468c3
174772025-04-30T06:13:43.878ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 23129,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:58020 req_id = dc72671c-44f5-484d-a535-a17d866468c3
174782025-04-30T06:13:43.879ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
174792025-04-30T06:13:43.879ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:58020 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = dc72671c-44f5-484d-a535-a17d866468c3
174802025-04-30T06:13:43.879ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = nat_v4_garbage_collector
174812025-04-30T06:13:43.901ZTRAComicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal
174822025-04-30T06:13:43.904ZTRAComicron-dev (dns): buffer buffer = Length: 72 (0x48) bytes\n0000: fd 99 01 00 00 01 00 00 00 00 00 01 09 5f 64 65 ............._de\n0010: 6e 64 72 69 74 65 04 5f 74 63 70 0d 63 6f 6e 74 ndrite._tcp.cont\n0020: 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 69 64 65 08 rol-plane.oxide.\n0030: 69 6e 74 65 72 6e 61 6c 00 00 21 00 01 00 00 29 internal..!....)\n0040: 04 d0 00 00 00 00 00 00 ........ kind = dns peer_addr = [::1]:62528 req_id = 1148f1c4-4691-49c9-b8e2-acf4d736a353
174832025-04-30T06:13:43.904ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 64921,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("_dendrite._tcp.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("_dendrite._tcp.control-plane.oxide.internal."),\n query_type: SRV,\n query_class: IN,\n },\n },\n original: [\n 9,\n 95,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 4,\n 95,\n 116,\n 99,\n 112,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 33,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:62528 req_id = 1148f1c4-4691-49c9-b8e2-acf4d736a353
174842025-04-30T06:13:43.905ZDEBGomicron-dev (store): query key key = _dendrite._tcp
174852025-04-30T06:13:43.905ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
174862025-04-30T06:13:43.905ZDEBGomicron-dev (dns): dns response additional_records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal"), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] kind = dns peer_addr = [::1]:62528 query = LowerQuery { name: LowerName(Name("_dendrite._tcp.control-plane.oxide.internal.")), original: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal."), query_type: SRV, query_class: IN } } records = [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal") })) }] req_id = 1148f1c4-4691-49c9-b8e2-acf4d736a353
174872025-04-30T06:13:43.905ZDEBGomicron-dev (DnsResolver): lookup_all_ipv6 srv dns_name = _dendrite._tcp.control-plane.oxide.internal response = SrvLookup(Lookup { query: Query { name: Name("_dendrite._tcp.control-plane.oxide.internal"), query_type: SRV, query_class: IN }, records: [Record { name_labels: Name("_dendrite._tcp.control-plane.oxide.internal."), rr_type: SRV, dns_class: IN, ttl: 0, rdata: Some(SRV(SRV { priority: 0, weight: 0, port: 46156, target: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.") })) }, Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }], valid_until: Instant { tv_sec: 2488, tv_nsec: 702634929 } })
174882025-04-30T06:13:43.905ZTRAComicron-dev (DnsResolver): lookup_service_targets: looking up SRV target
174892025-04-30T06:13:43.905ZTRAComicron-dev (dns): buffer buffer = Length: 108 (0x6c) bytes\n0000: 27 bd 01 00 00 01 00 00 00 00 00 01 2d 64 65 6e '...........-den\n0010: 64 72 69 74 65 2d 62 36 64 36 35 33 34 31 2d 31 drite-b6d65341-1\n0020: 36 37 63 2d 34 31 64 66 2d 39 62 35 63 2d 34 31 67c-41df-9b5c-41\n0030: 63 64 65 64 39 39 63 32 32 39 04 68 6f 73 74 0d cded99c229.host.\n0040: 63 6f 6e 74 72 6f 6c 2d 70 6c 61 6e 65 05 6f 78 control-plane.ox\n0050: 69 64 65 08 69 6e 74 65 72 6e 61 6c 00 00 1c 00 ide.internal....\n0060: 01 00 00 29 04 d0 00 00 00 00 00 00 ...)........ kind = dns peer_addr = [::1]:60105 req_id = aa082592-bc38-43fa-86a9-20ea9ef9db76
174902025-04-30T06:13:43.905ZDEBGomicron-dev (dns): message_request kind = dns mr = MessageRequest {\n header: Header {\n id: 10173,\n message_type: Query,\n op_code: Query,\n authoritative: false,\n truncation: false,\n recursion_desired: true,\n recursion_available: false,\n authentic_data: false,\n checking_disabled: false,\n response_code: NoError,\n query_count: 1,\n answer_count: 0,\n name_server_count: 0,\n additional_count: 1,\n },\n query: WireQuery {\n query: LowerQuery {\n name: LowerName(\n Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n ),\n original: Query {\n name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."),\n query_type: AAAA,\n query_class: IN,\n },\n },\n original: [\n 45,\n 100,\n 101,\n 110,\n 100,\n 114,\n 105,\n 116,\n 101,\n 45,\n 98,\n 54,\n 100,\n 54,\n 53,\n 51,\n 52,\n 49,\n 45,\n 49,\n 54,\n 55,\n 99,\n 45,\n 52,\n 49,\n 100,\n 102,\n 45,\n 57,\n 98,\n 53,\n 99,\n 45,\n 52,\n 49,\n 99,\n 100,\n 101,\n 100,\n 57,\n 57,\n 99,\n 50,\n 50,\n 57,\n 4,\n 104,\n 111,\n 115,\n 116,\n 13,\n 99,\n 111,\n 110,\n 116,\n 114,\n 111,\n 108,\n 45,\n 112,\n 108,\n 97,\n 110,\n 101,\n 5,\n 111,\n 120,\n 105,\n 100,\n 101,\n 8,\n 105,\n 110,\n 116,\n 101,\n 114,\n 110,\n 97,\n 108,\n 0,\n 0,\n 28,\n 0,\n 1,\n ],\n },\n answers: [],\n name_servers: [],\n additionals: [],\n sig0: [],\n edns: Some(\n Edns {\n rcode_high: 0,\n version: 0,\n dnssec_ok: false,\n max_payload: 1232,\n options: OPT {\n options: {},\n },\n },\n ),\n} peer_addr = [::1]:60105 req_id = aa082592-bc38-43fa-86a9-20ea9ef9db76
174912025-04-30T06:13:43.905ZDEBGomicron-dev (store): query key key = dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host
174922025-04-30T06:13:43.905ZDEBGomicron-dev (dns): dns response additional_records = [] kind = dns peer_addr = [::1]:60105 query = LowerQuery { name: LowerName(Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal.")), original: Query { name: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), query_type: AAAA, query_class: IN } } records = [Record { name_labels: Name("dendrite-b6d65341-167c-41df-9b5c-41cded99c229.host.control-plane.oxide.internal."), rr_type: AAAA, dns_class: IN, ttl: 0, rdata: Some(AAAA(AAAA(::1))) }] req_id = aa082592-bc38-43fa-86a9-20ea9ef9db76
174932025-04-30T06:13:43.905ZINFOomicron-dev (BackgroundTasks): Determining switch slots managed by switch zones background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
174942025-04-30T06:13:43.932ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = nat_v4_garbage_collector zone_address = ::1
174952025-04-30T06:13:43.932ZDEBGomicron-dev (MgsClient): client request background_task = nat_v4_garbage_collector body = None method = GET uri = http://[::1]:12225/local/switch-id
174962025-04-30T06:13:43.933ZDEBGomicron-dev (MgsClient): client response background_task = nat_v4_garbage_collector result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
174972025-04-30T06:13:43.933ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = nat_v4_garbage_collector reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
174982025-04-30T06:13:43.937ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = nat_v4_garbage_collector
174992025-04-30T06:13:43.952ZINFOomicron-dev (BackgroundTasks): determining switch slot managed by dendrite zone background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc zone_address = ::1
175002025-04-30T06:13:43.953ZDEBGomicron-dev (MgsClient): client request background_task = switch_port_config_manager body = None method = GET rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc uri = http://[::1]:12225/local/switch-id
175012025-04-30T06:13:43.953ZDEBGomicron-dev (MgsClient): client response background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc result = Err(reqwest::Error { kind: Request, url: "http://[::1]:12225/local/switch-id", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
175022025-04-30T06:13:43.953ZWARNomicron-dev (BackgroundTasks): failed to identify switch slot for dendrite background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc reason = Communication Error: error sending request for url (http://[::1]:12225/local/switch-id): client error (Connect): tcp connect error: Connection refused (os error 146): Connection refused (os error 146) zone_address = ::1
175032025-04-30T06:13:43.954ZWARNomicron-dev (BackgroundTasks): Failed to map switch zone addr: Communication Error: error sending request for url (http://[::1]:12225/local/switch-id), retrying background_task = switch_port_config_manager rack_id = c19a698f-c6f9-4a17-ae30-20d711b8f7dc
175042025-04-30T06:13:44.089ZDEBGomicron-dev (results-sink): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
175052025-04-30T06:13:44.089ZDEBGomicron-dev (clickhouse-client): unrolling 1 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
175062025-04-30T06:13:44.090ZDEBGomicron-dev (results-sink-cluster): inserting 1 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
175072025-04-30T06:13:44.090ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 2 sql = INSERT INTO oximeter.fields_string FORMAT Native
175082025-04-30T06:13:44.094ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 2 table_name = fields_string
175092025-04-30T06:13:44.094ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
175102025-04-30T06:13:44.102ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
175112025-04-30T06:13:44.102ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
175122025-04-30T06:13:44.192ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
175132025-04-30T06:13:44.679ZINFOomicron-dev (oximeter-agent): refreshing list of producers from Nexus collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 file = oximeter/collector/src/agent.rs:562
175142025-04-30T06:13:44.680ZDEBGomicron-dev (oximeter): client request body = None method = GET uri = http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
175152025-04-30T06:13:44.681ZTRAComicron-dev (dropshot_internal): incoming request local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 7a97f23b-9b6e-452e-b6e2-50401225d9f7 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
175162025-04-30T06:13:44.682ZTRAComicron-dev (dropshot_internal): authorize begin action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 7a97f23b-9b6e-452e-b6e2-50401225d9f7 resource = Database uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
175172025-04-30T06:13:44.683ZDEBGomicron-dev (dropshot_internal): roles actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 7a97f23b-9b6e-452e-b6e2-50401225d9f7 roles = RoleSet { roles: {} } uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
175182025-04-30T06:13:44.685ZDEBGomicron-dev (dropshot_internal): authorize result action = Query actor = Some(Actor::UserBuiltin { user_builtin_id: 001de000-05e4-4000-8000-000000000002, .. }) actor_id = 001de000-05e4-4000-8000-000000000002 authenticated = true local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 7a97f23b-9b6e-452e-b6e2-50401225d9f7 resource = Database result = Ok(()) uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
175192025-04-30T06:13:44.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175202025-04-30T06:13:44.719ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175212025-04-30T06:13:44.721ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175222025-04-30T06:13:44.721ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175232025-04-30T06:13:44.721ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175242025-04-30T06:13:44.721ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175252025-04-30T06:13:44.721ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
175262025-04-30T06:13:44.721ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
175272025-04-30T06:13:44.722ZTRAComicron-dev (dropshot): incoming request local_addr = [::1]:50290 method = GET remote_addr = [::1]:62154 req_id = d5f39fe6-336f-4b05-b413-4c4aed34af90 uri = /e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
175282025-04-30T06:13:44.725ZINFOomicron-dev (dropshot): request completed file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:867 latency_us = 3670 local_addr = [::1]:50290 method = GET remote_addr = [::1]:62154 req_id = d5f39fe6-336f-4b05-b413-4c4aed34af90 response_code = 200 uri = /e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
175292025-04-30T06:13:44.726ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
175302025-04-30T06:13:44.730ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 5 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
175312025-04-30T06:13:44.730ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
175322025-04-30T06:13:44.731ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
175332025-04-30T06:13:45.089ZDEBGomicron-dev (results-sink): inserting 6 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
175342025-04-30T06:13:45.089ZDEBGomicron-dev (clickhouse-client): unrolling 6 total samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e
175352025-04-30T06:13:45.091ZDEBGomicron-dev (results-sink-cluster): inserting 6 samples into database collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
175362025-04-30T06:13:45.095ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 12 sql = INSERT INTO oximeter.fields_string FORMAT Native
175372025-04-30T06:13:45.099ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 12 table_name = fields_string
175382025-04-30T06:13:45.099ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 5 sql = INSERT INTO oximeter.fields_u16 FORMAT Native
175392025-04-30T06:13:45.103ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 5 table_name = fields_u16
175402025-04-30T06:13:45.103ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 5 sql = INSERT INTO oximeter.fields_uuid FORMAT Native
175412025-04-30T06:13:45.107ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 5 table_name = fields_uuid
175422025-04-30T06:13:45.107ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 19 n_rows = 5 sql = INSERT INTO oximeter.measurements_histogramu64 FORMAT Native
175432025-04-30T06:13:45.110ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 5 table_name = measurements_histogramu64
175442025-04-30T06:13:45.110ZTRAComicron-dev (clickhouse-client): inserting data collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_columns = 4 n_rows = 1 sql = INSERT INTO oximeter.measurements_i64 FORMAT Native
175452025-04-30T06:13:45.114ZDEBGomicron-dev (clickhouse-client): inserted rows into table collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 id = 5d3e73c4-6821-4f57-9b76-9340bc18294e n_rows = 1 table_name = measurements_i64
175462025-04-30T06:13:45.114ZTRAComicron-dev (results-sink): successfully inserted samples collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1
175472025-04-30T06:13:45.193ZWARNomicron-dev (results-sink-cluster): failed to insert some results into metric DB collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
175482025-04-30T06:13:45.718ZDEBGomicron-dev (collection-task): sent timer-based collection request to the collection task collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175492025-04-30T06:13:45.718ZDEBGomicron-dev (collection-task): collecting from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175502025-04-30T06:13:45.720ZTRAComicron-dev (collection-task): sent collection request to producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175512025-04-30T06:13:45.720ZDEBGomicron-dev (collection-task): collected results from producer collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 n_results = 1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175522025-04-30T06:13:45.720ZTRAComicron-dev (collection-task): forwarded results to main collection loop collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175532025-04-30T06:13:45.720ZTRAComicron-dev (collection-task): top of inner collection loop, waiting for next request collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 producer_id = a6458b7d-87c3-4483-be96-854d814c20de
175542025-04-30T06:13:45.755ZWARNomicron-dev (BackgroundTasks): inventory collection failed background_task = inventory_collection error = failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint "inv_caboose_pkey"
175552025-04-30T06:13:45.755ZDEBGomicron-dev (BackgroundTasks): activation complete background_task = inventory_collection elapsed = 2.787419342s iteration = 66 status = {"error":"failed to collect inventory: saving inventory to database: Internal Error: unexpected database error: duplicate key value violates unique constraint \\"inv_caboose_pkey\\""}
175562025-04-30T06:13:45.763ZINFOomicron-dev (dropshot_internal): request completed error_message_external = Internal Server Error error_message_internal = database error (kind = ClosedConnection): server closed the connection unexpectedly\n\tThis probably means the server terminated abnormally\n\tbefore or while processing the request.\n\n latency_us = 1082366 local_addr = [::1]:12221 method = GET remote_addr = [::1]:40743 req_id = 7a97f23b-9b6e-452e-b6e2-50401225d9f7 response_code = 500 uri = /metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending
175572025-04-30T06:13:45.764ZDEBGomicron-dev (oximeter): client response result = Ok(Response { url: "http://[::1]:12221/metrics/collectors/39e6175b-4df2-4730-b11d-cbc1e60a2e78/producers?sort_by=id_ascending", status: 500, headers: {"content-type": "application/json", "x-request-id": "7a97f23b-9b6e-452e-b6e2-50401225d9f7", "content-length": "124", "date": "Wed, 30 Apr 2025 06:13:45 GMT"} })
175582025-04-30T06:13:45.764ZERROomicron-dev (oximeter-agent): error fetching next assigned producer, abandoning this refresh attempt collector_id = 39e6175b-4df2-4730-b11d-cbc1e60a2e78 collector_ip = ::1 error = Error Response: status: 500 Internal Server Error; headers: {"content-type": "application/json", "x-request-id": "7a97f23b-9b6e-452e-b6e2-50401225d9f7", "content-length": "124", "date": "Wed, 30 Apr 2025 06:13:45 GMT"}; value: Error { error_code: Some("Internal"), message: "Internal Server Error", request_id: "7a97f23b-9b6e-452e-b6e2-50401225d9f7" } file = oximeter/collector/src/agent.rs:600
175592025-04-30T06:13:45.764ZWARNomicron-dev (ServerContext): Failed is_valid check; connection failed ping error = no connection to the server\n
175602025-04-30T06:13:45.766ZWARNomicron-dev (ServerContext): Failed to make connection backend = [::1]:60242 error = could not connect to server: Connection refused\n\tIs the server running on host "::1" and accepting\n\tTCP/IP connections on port 60242?\n
175612025-04-30T06:13:45.821ZWARNomicron-dev (ServerContext): Failed to make connection backend = [::1]:60242 error = could not connect to server: Connection refused\n\tIs the server running on host "::1" and accepting\n\tTCP/IP connections on port 60242?\n
175622025-04-30T06:13:45.847ZINFOomicron-dev (dropshot): beginning graceful shutdown local_addr = [::1]:61881 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
175632025-04-30T06:13:45.849ZINFOomicron-dev (dropshot (Repo Depot)): beginning graceful shutdown local_addr = [::1]:43978 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
175642025-04-30T06:13:45.850ZWARNomicron-dev (ArtifactStore): All ledger manager request senders dropped server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
175652025-04-30T06:13:45.850ZWARNomicron-dev (ArtifactStore): Delete reconciler sender dropped server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
175662025-04-30T06:13:45.851ZINFOomicron-dev (dropshot): beginning graceful shutdown kind = pantry local_addr = [::1]:55020 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
175672025-04-30T06:13:45.851ZINFOomicron-dev (dropshot): beginning graceful shutdown file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.0/src/server.rs:317 local_addr = [::1]:41936
175682025-04-30T06:13:45.852ZINFOomicron-dev (Simulated CrucibleAgent Dropshot Server): beginning graceful shutdown kind = storage local_addr = [::1]:34446 server = b6d65341-167c-41df-9b5c-41cded99c229 sled_id = b6d65341-167c-41df-9b5c-41cded99c229
175692025-04-30T06:13:45.871ZINFOomicron-dev (http): beginning graceful shutdown kind = dns local_addr = [::1]:39475