Max level shown:
1 [ May 29 19:43:30 Enabled. ]
2 [ May 29 19:43:30 Rereading configuration. ]
3 [ May 29 19:43:31 Rereading configuration. ]
4 [ May 29 19:43:34 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/oximeter-collector/bin/oximeter run /var/svc/manifest/site/oximeter/config.toml --address [fd00:1122:3344:101::d]:12223 --id 901ec26b-90dc-4813-8833-d7a056f32f59 &"). ]
5 [ May 29 19:43:34 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72026-05-29T19:43:35.015ZDEBGoximeter: registered DTrace probes
82026-05-29T19:43:35.017ZINFOoximeter: starting oximeter server file = oximeter/collector/src/lib.rs:301
92026-05-29T19:43:35.017ZDEBGoximeter: creating ClickHouse client
102026-05-29T19:43:35.052ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
112026-05-29T19:43:41.409ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:890 id = 73d9de17-7b01-4188-88a3-5b079937dd77
122026-05-29T19:43:41.409ZDEBGoximeter (oximeter-agent): oximeter database does not exist, creating collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d
132026-05-29T19:43:41.409ZINFOoximeter (clickhouse-client): reading db version collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:790 id = 73d9de17-7b01-4188-88a3-5b079937dd77
142026-05-29T19:43:41.410ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:890 id = 73d9de17-7b01-4188-88a3-5b079937dd77
152026-05-29T19:43:41.410ZINFOoximeter (clickhouse-client): read oximeter database version collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:795 id = 73d9de17-7b01-4188-88a3-5b079937dd77 version = 0
162026-05-29T19:43:41.410ZINFOoximeter (clickhouse-client): initializing oximeter schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:800 id = 73d9de17-7b01-4188-88a3-5b079937dd77
172026-05-29T19:43:41.410ZDEBGoximeter (clickhouse-client): initializing ClickHouse database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
182026-05-29T19:43:41.519ZINFOoximeter (clickhouse-client): inserting current version collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:821 id = 73d9de17-7b01-4188-88a3-5b079937dd77 version = 14
192026-05-29T19:43:41.542ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 7ff13152-d803-4e50-9c5e-b3d2717f851a
202026-05-29T19:43:41.564ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /info versions = all
212026-05-29T19:43:41.564ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers versions = all
222026-05-29T19:43:41.564ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = DELETE path = /producers/{producer_id} versions = all
232026-05-29T19:43:41.564ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers/{producer_id} versions = all
242026-05-29T19:43:41.564ZINFOoximeter (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.0/src/server.rs:335 local_addr = [fd00:1122:3344:101::d]:12223
252026-05-29T19:43:41.569ZDEBGoximeter (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::d]:12223
262026-05-29T19:43:41.569ZDEBGoximeter: registered USDT probes
272026-05-29T19:43:41.569ZDEBGoximeter: contacting nexus
282026-05-29T19:43:41.599ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors
292026-05-29T19:43:41.768ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors", status: 204, headers: {"x-request-id": "72827c47-8b8f-49d5-b3aa-5b87cbbac189", "date": "Fri, 29 May 2026 19:43:41 GMT"} })
302026-05-29T19:43:41.768ZINFOoximeter: oximeter registered with nexus file = oximeter/collector/src/lib.rs:464 id = 901ec26b-90dc-4813-8833-d7a056f32f59
312026-05-29T19:43:41.768ZINFOoximeter (oximeter-agent): starting refresh list task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:562
322026-05-29T19:43:41.769ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
332026-05-29T19:43:41.772ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending
342026-05-29T19:43:41.886ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "4f6f9c53-77eb-4130-b0e7-caedc164fac3", "content-length": "29", "date": "Fri, 29 May 2026 19:43:41 GMT"} })
352026-05-29T19:43:41.886ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 0 n_pruned_tasks = 0
362026-05-29T19:43:46.544ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d
372026-05-29T19:43:46.544ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d
382026-05-29T19:43:51.545ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d
392026-05-29T19:43:51.545ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d
402026-05-29T19:43:56.545ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d
412026-05-29T19:43:56.546ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d
422026-05-29T19:43:56.771ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
432026-05-29T19:43:56.771ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending
442026-05-29T19:43:56.784ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "4139986e-753e-46ae-9611-263f67a1a7d2", "content-length": "436", "date": "Fri, 29 May 2026 19:43:56 GMT"} })
452026-05-29T19:43:56.786ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiNTUxZDBiZS03NjA2LTQ4ODEtODU1My1lNDg0MDE1YjJmYTAifX0%3D
462026-05-29T19:43:56.790ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiNTUxZDBiZS03NjA2LTQ4ODEtODU1My1lNDg0MDE1YjJmYTAifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "4cd1eab9-530d-409c-802a-613993c69947", "content-length": "29", "date": "Fri, 29 May 2026 19:43:56 GMT"} })
472026-05-29T19:43:56.790ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:8001 collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad
482026-05-29T19:43:56.790ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:56326 collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0
492026-05-29T19:43:56.790ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 2 n_pruned_tasks = 0
502026-05-29T19:43:56.791ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
512026-05-29T19:43:56.791ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
522026-05-29T19:43:56.791ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
532026-05-29T19:43:56.791ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
542026-05-29T19:43:56.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
552026-05-29T19:43:56.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
562026-05-29T19:43:56.817ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
572026-05-29T19:43:56.823ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
582026-05-29T19:43:57.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
592026-05-29T19:43:57.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
602026-05-29T19:43:57.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
612026-05-29T19:43:58.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
622026-05-29T19:43:58.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
632026-05-29T19:43:58.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
642026-05-29T19:43:59.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
652026-05-29T19:43:59.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
662026-05-29T19:43:59.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
672026-05-29T19:44:00.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
682026-05-29T19:44:00.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
692026-05-29T19:44:00.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
702026-05-29T19:44:01.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 212
712026-05-29T19:44:01.546ZDEBGoximeter (clickhouse-client): unrolling 212 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
722026-05-29T19:44:01.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 212
732026-05-29T19:44:01.556ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
742026-05-29T19:44:01.566ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550108505Z } timeseries_name = ddm_router:originated_tunnel_endpoints
752026-05-29T19:44:01.566ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550012374Z } timeseries_name = ddm_router:originated_underlay_prefixes
762026-05-29T19:44:01.566ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550363939Z } timeseries_name = ddm_session:advertisements_received
772026-05-29T19:44:01.566ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550301918Z } timeseries_name = ddm_session:advertisements_sent
782026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550843456Z } timeseries_name = ddm_session:imported_tunnel_endpoints
792026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550783515Z } timeseries_name = ddm_session:imported_underlay_prefixes
802026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_address_changes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550484161Z } timeseries_name = ddm_session:peer_address_changes
812026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_expirations"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550423750Z } timeseries_name = ddm_session:peer_expirations
822026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_sessions_established"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550543822Z } timeseries_name = ddm_session:peer_sessions_established
832026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550234967Z } timeseries_name = ddm_session:solicitations_received
842026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550172456Z } timeseries_name = ddm_session:solicitations_sent
852026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:update_send_fail"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550724184Z } timeseries_name = ddm_session:update_send_fail
862026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550664013Z } timeseries_name = ddm_session:updates_received
872026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550604463Z } timeseries_name = ddm_session:updates_sent
882026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("dendrite:sample_collection_duration"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Metric, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Metric, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Metric, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Metric, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Metric, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Metric, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Metric, description: "" }}, datum_type: F64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.555235740Z } timeseries_name = dendrite:sample_collection_duration
892026-05-29T19:44:01.567ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:collections"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "base_route", field_type: String, source: Metric, description: "" }, FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "producer_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "producer_ip", field_type: IpAddr, source: Metric, description: "" }, FieldSchema { name: "producer_port", field_type: U16, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.549811311Z } timeseries_name = oximeter_collector:collections
902026-05-29T19:44:01.568ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_data_link:link_enabled"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_id", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "port_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: Bool, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550907387Z } timeseries_name = switch_data_link:link_enabled
912026-05-29T19:44:01.568ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_data_link:link_up"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_id", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "port_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: Bool, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.550972528Z } timeseries_name = switch_data_link:link_up
922026-05-29T19:44:01.568ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:capacity"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.551093910Z } timeseries_name = switch_table:capacity
932026-05-29T19:44:01.568ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:collisions"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.551411554Z } timeseries_name = switch_table:collisions
942026-05-29T19:44:01.568ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:delete_misses"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.551536366Z } timeseries_name = switch_table:delete_misses
952026-05-29T19:44:01.568ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:deletes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.551286893Z } timeseries_name = switch_table:deletes
962026-05-29T19:44:01.568ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:exhaustion"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.551599767Z } timeseries_name = switch_table:exhaustion
972026-05-29T19:44:01.568ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:inserts"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.551223501Z } timeseries_name = switch_table:inserts
982026-05-29T19:44:01.568ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:occupancy"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.551162661Z } timeseries_name = switch_table:occupancy
992026-05-29T19:44:01.569ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:update_misses"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.551474215Z } timeseries_name = switch_table:update_misses
1002026-05-29T19:44:01.569ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:updates"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:01.551349243Z } timeseries_name = switch_table:updates
1012026-05-29T19:44:01.571ZDEBGoximeter (clickhouse-client): inserting 27 new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
1022026-05-29T19:44:01.579ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 280 table_name = fields_i16
1032026-05-29T19:44:01.580ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 4 table_name = fields_ipaddr
1042026-05-29T19:44:01.582ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 749 table_name = fields_string
1052026-05-29T19:44:01.584ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 144 table_name = fields_u16
1062026-05-29T19:44:01.586ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 145 table_name = fields_u32
1072026-05-29T19:44:01.587ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 144 table_name = fields_u8
1082026-05-29T19:44:01.589ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 452 table_name = fields_uuid
1092026-05-29T19:44:01.591ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 4 table_name = measurements_bool
1102026-05-29T19:44:01.592ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 157 table_name = measurements_cumulativeu64
1112026-05-29T19:44:01.594ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1 table_name = measurements_f64
1122026-05-29T19:44:01.596ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_u64
1132026-05-29T19:44:01.648ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
1142026-05-29T19:44:01.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1152026-05-29T19:44:01.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1162026-05-29T19:44:01.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1172026-05-29T19:44:02.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1182026-05-29T19:44:02.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1192026-05-29T19:44:02.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1202026-05-29T19:44:03.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1212026-05-29T19:44:03.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1222026-05-29T19:44:03.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1232026-05-29T19:44:04.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1242026-05-29T19:44:04.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1252026-05-29T19:44:04.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1262026-05-29T19:44:05.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1272026-05-29T19:44:05.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1282026-05-29T19:44:05.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1292026-05-29T19:44:06.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 70
1302026-05-29T19:44:06.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 70
1312026-05-29T19:44:06.546ZDEBGoximeter (clickhouse-client): unrolling 70 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
1322026-05-29T19:44:06.549ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
1332026-05-29T19:44:06.570ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 26 table_name = fields_string
1342026-05-29T19:44:06.572ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 28 table_name = fields_uuid
1352026-05-29T19:44:06.573ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_cumulativeu64
1362026-05-29T19:44:06.576ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 20 table_name = measurements_u64
1372026-05-29T19:44:06.646ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
1382026-05-29T19:44:06.790ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
1392026-05-29T19:44:06.790ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
1402026-05-29T19:44:06.803ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
1412026-05-29T19:44:06.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1422026-05-29T19:44:06.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1432026-05-29T19:44:06.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1442026-05-29T19:44:07.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1452026-05-29T19:44:07.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1462026-05-29T19:44:07.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1472026-05-29T19:44:08.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1482026-05-29T19:44:08.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1492026-05-29T19:44:08.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1502026-05-29T19:44:09.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1512026-05-29T19:44:09.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1522026-05-29T19:44:09.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1532026-05-29T19:44:10.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1542026-05-29T19:44:10.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1552026-05-29T19:44:10.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1562026-05-29T19:44:11.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
1572026-05-29T19:44:11.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
1582026-05-29T19:44:11.546ZDEBGoximeter (clickhouse-client): unrolling 210 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
1592026-05-29T19:44:11.554ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
1602026-05-29T19:44:11.563ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 280 table_name = fields_i16
1612026-05-29T19:44:11.565ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 747 table_name = fields_string
1622026-05-29T19:44:11.567ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 140 table_name = fields_u16
1632026-05-29T19:44:11.569ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 145 table_name = fields_u32
1642026-05-29T19:44:11.571ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 144 table_name = fields_u8
1652026-05-29T19:44:11.573ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 448 table_name = fields_uuid
1662026-05-29T19:44:11.574ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 4 table_name = measurements_bool
1672026-05-29T19:44:11.576ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 155 table_name = measurements_cumulativeu64
1682026-05-29T19:44:11.578ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1 table_name = measurements_f64
1692026-05-29T19:44:11.580ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_u64
1702026-05-29T19:44:11.647ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
1712026-05-29T19:44:11.771ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
1722026-05-29T19:44:11.771ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending
1732026-05-29T19:44:11.794ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "6f9153f8-df09-4aec-9825-39321b78b228", "content-length": "715", "date": "Fri, 29 May 2026 19:44:11 GMT"} })
1742026-05-29T19:44:11.794ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiNTUxZDBiZS03NjA2LTQ4ODEtODU1My1lNDg0MDE1YjJmYTAifX0%3D
1752026-05-29T19:44:11.797ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiNTUxZDBiZS03NjA2LTQ4ODEtODU1My1lNDg0MDE1YjJmYTAifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "4a975a10-392e-4f6e-9470-3f30749393e5", "content-length": "29", "date": "Fri, 29 May 2026 19:44:11 GMT"} })
1762026-05-29T19:44:11.797ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::b]:43728 collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377
1772026-05-29T19:44:11.797ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:37744 collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 22fdeb8b-488b-4808-ad25-4b8f01f41850
1782026-05-29T19:44:11.797ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 4 n_pruned_tasks = 0
1792026-05-29T19:44:11.798ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 22fdeb8b-488b-4808-ad25-4b8f01f41850 producer_ip = fd00:1122:3344:101::1 producer_port = 37744
1802026-05-29T19:44:11.799ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 22fdeb8b-488b-4808-ad25-4b8f01f41850 producer_ip = fd00:1122:3344:101::1 producer_port = 37744
1812026-05-29T19:44:11.799ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
1822026-05-29T19:44:11.799ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 22fdeb8b-488b-4808-ad25-4b8f01f41850 producer_ip = fd00:1122:3344:101::1 producer_port = 37744
1832026-05-29T19:44:11.799ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
1842026-05-29T19:44:11.799ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
1852026-05-29T19:44:11.807ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 12 producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
1862026-05-29T19:44:11.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1872026-05-29T19:44:11.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1882026-05-29T19:44:11.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1892026-05-29T19:44:11.947ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 3 n_samples = 2543 producer_id = 22fdeb8b-488b-4808-ad25-4b8f01f41850 producer_ip = fd00:1122:3344:101::1 producer_port = 37744
1902026-05-29T19:44:11.963ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 2571
1912026-05-29T19:44:11.963ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 2571
1922026-05-29T19:44:11.963ZDEBGoximeter (clickhouse-client): unrolling 2571 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
1932026-05-29T19:44:12.051ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
1942026-05-29T19:44:12.054ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:cpus_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.964223136Z } timeseries_name = collection_target:cpus_provisioned
1952026-05-29T19:44:12.054ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:ram_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.964300237Z } timeseries_name = collection_target:ram_provisioned
1962026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:virtual_disk_space_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.964373658Z } timeseries_name = collection_target:virtual_disk_space_provisioned
1972026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.964079374Z } timeseries_name = http_service:request_latency_histogram
1982026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("kstat_sampler:expired_targets"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:12.039625563Z } timeseries_name = kstat_sampler:expired_targets
1992026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("kstat_sampler:samples_dropped"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "target_id", field_type: U64, source: Metric, description: "" }, FieldSchema { name: "target_name", field_type: String, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:12.039881187Z } timeseries_name = kstat_sampler:samples_dropped
2002026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("sled_cpu:cpu_nsec"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "cpu_id", field_type: U32, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "state", field_type: String, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:12.005365225Z } timeseries_name = sled_cpu:cpu_nsec
2012026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("sled_data_link:bytes_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "zone_name", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.965910811Z } timeseries_name = sled_data_link:bytes_received
2022026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("sled_data_link:bytes_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "zone_name", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.966090854Z } timeseries_name = sled_data_link:bytes_sent
2032026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("sled_data_link:errors_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "zone_name", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.965741089Z } timeseries_name = sled_data_link:errors_received
2042026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("sled_data_link:errors_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "zone_name", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.965814299Z } timeseries_name = sled_data_link:errors_sent
2052026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("sled_data_link:packets_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "zone_name", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.966023142Z } timeseries_name = sled_data_link:packets_received
2062026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("sled_data_link:packets_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "zone_name", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.966156424Z } timeseries_name = sled_data_link:packets_sent
2072026-05-29T19:44:12.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("zfs_dataset:bytes_used"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "dataset_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "dataset_kind", field_type: String, source: Metric, description: "" }, FieldSchema { name: "dataset_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "pool_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "pool_kind", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "zone_name", field_type: String, source: Metric, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:12.041473240Z } timeseries_name = zfs_dataset:bytes_used
2082026-05-29T19:44:12.056ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("zfs_pool:bytes_allocated"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "pool_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "pool_kind", field_type: String, source: Metric, description: "" }, FieldSchema { name: "pool_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:12.040364204Z } timeseries_name = zfs_pool:bytes_allocated
2092026-05-29T19:44:12.056ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("zfs_pool:bytes_total"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "pool_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "pool_kind", field_type: String, source: Metric, description: "" }, FieldSchema { name: "pool_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:12.040442255Z } timeseries_name = zfs_pool:bytes_total
2102026-05-29T19:44:12.056ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 schema = TimeseriesSchema { timeseries_name: TimeseriesName("zone:cpu_nsec"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "state", field_type: String, source: Metric, description: "" }, FieldSchema { name: "zone_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "zone_name", field_type: String, source: Metric, description: "" }, FieldSchema { name: "zone_type", field_type: String, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-05-29T19:44:11.980525544Z } timeseries_name = zone:cpu_nsec
2112026-05-29T19:44:12.064ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
2122026-05-29T19:44:12.073ZDEBGoximeter (clickhouse-client): inserting 17 new timeseries schema collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
2132026-05-29T19:44:12.079ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 4 table_name = fields_ipaddr
2142026-05-29T19:44:12.084ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 3230 table_name = fields_string
2152026-05-29T19:44:12.086ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 10 table_name = fields_u16
2162026-05-29T19:44:12.089ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 953 table_name = fields_u32
2172026-05-29T19:44:12.091ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 2 table_name = fields_u64
2182026-05-29T19:44:12.094ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1844 table_name = fields_uuid
2192026-05-29T19:44:12.097ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 2248 table_name = measurements_cumulativeu64
2202026-05-29T19:44:12.102ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 6 table_name = measurements_histogramu64
2212026-05-29T19:44:12.103ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 9 table_name = measurements_i64
2222026-05-29T19:44:12.105ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 308 table_name = measurements_u64
2232026-05-29T19:44:12.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2242026-05-29T19:44:12.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2252026-05-29T19:44:12.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2262026-05-29T19:44:13.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2272026-05-29T19:44:13.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2282026-05-29T19:44:13.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2292026-05-29T19:44:14.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2302026-05-29T19:44:14.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2312026-05-29T19:44:14.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2322026-05-29T19:44:15.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2332026-05-29T19:44:15.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2342026-05-29T19:44:15.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2352026-05-29T19:44:16.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 56
2362026-05-29T19:44:16.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 56
2372026-05-29T19:44:16.546ZDEBGoximeter (clickhouse-client): unrolling 56 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
2382026-05-29T19:44:16.548ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
2392026-05-29T19:44:16.561ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 26 table_name = fields_string
2402026-05-29T19:44:16.563ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 28 table_name = fields_uuid
2412026-05-29T19:44:16.564ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 40 table_name = measurements_cumulativeu64
2422026-05-29T19:44:16.566ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 16 table_name = measurements_u64
2432026-05-29T19:44:16.646ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
2442026-05-29T19:44:16.791ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
2452026-05-29T19:44:16.791ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
2462026-05-29T19:44:16.803ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
2472026-05-29T19:44:16.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2482026-05-29T19:44:16.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2492026-05-29T19:44:16.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2502026-05-29T19:44:17.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2512026-05-29T19:44:17.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2522026-05-29T19:44:17.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2532026-05-29T19:44:18.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2542026-05-29T19:44:18.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2552026-05-29T19:44:18.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2562026-05-29T19:44:19.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2572026-05-29T19:44:19.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2582026-05-29T19:44:19.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2592026-05-29T19:44:20.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2602026-05-29T19:44:20.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2612026-05-29T19:44:20.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2622026-05-29T19:44:21.545ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
2632026-05-29T19:44:21.545ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
2642026-05-29T19:44:21.545ZDEBGoximeter (clickhouse-client): unrolling 210 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
2652026-05-29T19:44:21.553ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
2662026-05-29T19:44:21.562ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 280 table_name = fields_i16
2672026-05-29T19:44:21.564ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 747 table_name = fields_string
2682026-05-29T19:44:21.565ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 140 table_name = fields_u16
2692026-05-29T19:44:21.567ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 145 table_name = fields_u32
2702026-05-29T19:44:21.569ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 144 table_name = fields_u8
2712026-05-29T19:44:21.571ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 448 table_name = fields_uuid
2722026-05-29T19:44:21.573ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 4 table_name = measurements_bool
2732026-05-29T19:44:21.574ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 155 table_name = measurements_cumulativeu64
2742026-05-29T19:44:21.576ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1 table_name = measurements_f64
2752026-05-29T19:44:21.578ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_u64
2762026-05-29T19:44:21.647ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
2772026-05-29T19:44:21.798ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
2782026-05-29T19:44:21.798ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
2792026-05-29T19:44:21.802ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 36 producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
2802026-05-29T19:44:21.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2812026-05-29T19:44:21.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2822026-05-29T19:44:21.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2832026-05-29T19:44:22.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2842026-05-29T19:44:22.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2852026-05-29T19:44:22.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2862026-05-29T19:44:23.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2872026-05-29T19:44:23.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2882026-05-29T19:44:23.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2892026-05-29T19:44:24.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2902026-05-29T19:44:24.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2912026-05-29T19:44:24.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2922026-05-29T19:44:25.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2932026-05-29T19:44:25.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2942026-05-29T19:44:25.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2952026-05-29T19:44:26.545ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 106
2962026-05-29T19:44:26.545ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 106
2972026-05-29T19:44:26.545ZDEBGoximeter (clickhouse-client): unrolling 106 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
2982026-05-29T19:44:26.549ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
2992026-05-29T19:44:26.555ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 68 table_name = fields_string
3002026-05-29T19:44:26.557ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 21 table_name = fields_u16
3012026-05-29T19:44:26.559ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 57 table_name = fields_uuid
3022026-05-29T19:44:26.561ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_cumulativeu64
3032026-05-29T19:44:26.563ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 21 table_name = measurements_histogramu64
3042026-05-29T19:44:26.565ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 15 table_name = measurements_i64
3052026-05-29T19:44:26.566ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 20 table_name = measurements_u64
3062026-05-29T19:44:26.646ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
3072026-05-29T19:44:26.770ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
3082026-05-29T19:44:26.770ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending
3092026-05-29T19:44:26.774ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "4bf05500-a74f-4ee9-99d0-556dbadb283c", "content-length": "715", "date": "Fri, 29 May 2026 19:44:26 GMT"} })
3102026-05-29T19:44:26.774ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiNTUxZDBiZS03NjA2LTQ4ODEtODU1My1lNDg0MDE1YjJmYTAifX0%3D
3112026-05-29T19:44:26.777ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiNTUxZDBiZS03NjA2LTQ4ODEtODU1My1lNDg0MDE1YjJmYTAifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "875f2bc4-e225-4038-829a-babf0fc359ad", "content-length": "29", "date": "Fri, 29 May 2026 19:44:26 GMT"} })
3122026-05-29T19:44:26.777ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 4 n_pruned_tasks = 0
3132026-05-29T19:44:26.791ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
3142026-05-29T19:44:26.791ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
3152026-05-29T19:44:26.803ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
3162026-05-29T19:44:26.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3172026-05-29T19:44:26.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3182026-05-29T19:44:26.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3192026-05-29T19:44:27.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3202026-05-29T19:44:27.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3212026-05-29T19:44:27.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3222026-05-29T19:44:28.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3232026-05-29T19:44:28.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3242026-05-29T19:44:28.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3252026-05-29T19:44:29.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3262026-05-29T19:44:29.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3272026-05-29T19:44:29.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3282026-05-29T19:44:30.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3292026-05-29T19:44:30.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3302026-05-29T19:44:30.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3312026-05-29T19:44:31.545ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
3322026-05-29T19:44:31.545ZDEBGoximeter (clickhouse-client): unrolling 210 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
3332026-05-29T19:44:31.545ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
3342026-05-29T19:44:31.553ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
3352026-05-29T19:44:31.561ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 280 table_name = fields_i16
3362026-05-29T19:44:31.563ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 747 table_name = fields_string
3372026-05-29T19:44:31.565ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 140 table_name = fields_u16
3382026-05-29T19:44:31.567ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 145 table_name = fields_u32
3392026-05-29T19:44:31.569ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 144 table_name = fields_u8
3402026-05-29T19:44:31.571ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 448 table_name = fields_uuid
3412026-05-29T19:44:31.572ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 4 table_name = measurements_bool
3422026-05-29T19:44:31.574ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 155 table_name = measurements_cumulativeu64
3432026-05-29T19:44:31.575ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1 table_name = measurements_f64
3442026-05-29T19:44:31.577ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_u64
3452026-05-29T19:44:31.646ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
3462026-05-29T19:44:31.798ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
3472026-05-29T19:44:31.798ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
3482026-05-29T19:44:31.802ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 23 producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
3492026-05-29T19:44:31.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3502026-05-29T19:44:31.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3512026-05-29T19:44:31.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3522026-05-29T19:44:32.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3532026-05-29T19:44:32.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3542026-05-29T19:44:32.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3552026-05-29T19:44:33.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3562026-05-29T19:44:33.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3572026-05-29T19:44:33.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3582026-05-29T19:44:34.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3592026-05-29T19:44:34.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3602026-05-29T19:44:34.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3612026-05-29T19:44:35.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3622026-05-29T19:44:35.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3632026-05-29T19:44:35.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3642026-05-29T19:44:36.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 93
3652026-05-29T19:44:36.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 93
3662026-05-29T19:44:36.546ZDEBGoximeter (clickhouse-client): unrolling 93 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
3672026-05-29T19:44:36.549ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
3682026-05-29T19:44:36.555ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 72 table_name = fields_string
3692026-05-29T19:44:36.557ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 23 table_name = fields_u16
3702026-05-29T19:44:36.559ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 51 table_name = fields_uuid
3712026-05-29T19:44:36.560ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_cumulativeu64
3722026-05-29T19:44:36.563ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 23 table_name = measurements_histogramu64
3732026-05-29T19:44:36.564ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 20 table_name = measurements_u64
3742026-05-29T19:44:36.647ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
3752026-05-29T19:44:36.791ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
3762026-05-29T19:44:36.791ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
3772026-05-29T19:44:36.802ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
3782026-05-29T19:44:36.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3792026-05-29T19:44:36.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3802026-05-29T19:44:36.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3812026-05-29T19:44:37.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3822026-05-29T19:44:37.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3832026-05-29T19:44:37.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3842026-05-29T19:44:38.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3852026-05-29T19:44:38.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3862026-05-29T19:44:38.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3872026-05-29T19:44:39.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3882026-05-29T19:44:39.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3892026-05-29T19:44:39.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3902026-05-29T19:44:40.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3912026-05-29T19:44:40.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3922026-05-29T19:44:40.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3932026-05-29T19:44:41.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
3942026-05-29T19:44:41.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
3952026-05-29T19:44:41.546ZDEBGoximeter (clickhouse-client): unrolling 210 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
3962026-05-29T19:44:41.553ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
3972026-05-29T19:44:41.561ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 280 table_name = fields_i16
3982026-05-29T19:44:41.563ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 747 table_name = fields_string
3992026-05-29T19:44:41.565ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 140 table_name = fields_u16
4002026-05-29T19:44:41.566ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 145 table_name = fields_u32
4012026-05-29T19:44:41.568ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 144 table_name = fields_u8
4022026-05-29T19:44:41.569ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 448 table_name = fields_uuid
4032026-05-29T19:44:41.570ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 4 table_name = measurements_bool
4042026-05-29T19:44:41.571ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 155 table_name = measurements_cumulativeu64
4052026-05-29T19:44:41.573ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1 table_name = measurements_f64
4062026-05-29T19:44:41.574ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_u64
4072026-05-29T19:44:41.648ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
4082026-05-29T19:44:41.771ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
4092026-05-29T19:44:41.771ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending
4102026-05-29T19:44:41.775ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "1de521fe-410b-41ea-9e8f-bf7c96b4e91c", "content-length": "864", "date": "Fri, 29 May 2026 19:44:41 GMT"} })
4112026-05-29T19:44:41.775ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODMyYWJlYy0xMDg0LTRjYmQtODA0My0xODZlNzBlODYyZjcifX0%3D
4122026-05-29T19:44:41.778ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODMyYWJlYy0xMDg0LTRjYmQtODA0My0xODZlNzBlODYyZjcifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "b6525ee0-d7bf-4066-b274-6e3425dd4c90", "content-length": "29", "date": "Fri, 29 May 2026 19:44:41 GMT"} })
4132026-05-29T19:44:41.778ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:36187 collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = f832abec-1084-4cbd-8043-186e70e862f7
4142026-05-29T19:44:41.778ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 5 n_pruned_tasks = 0
4152026-05-29T19:44:41.779ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = f832abec-1084-4cbd-8043-186e70e862f7 producer_ip = fd00:1122:3344:101::2 producer_port = 36187
4162026-05-29T19:44:41.780ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = f832abec-1084-4cbd-8043-186e70e862f7 producer_ip = fd00:1122:3344:101::2 producer_port = 36187
4172026-05-29T19:44:41.780ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = f832abec-1084-4cbd-8043-186e70e862f7 producer_ip = fd00:1122:3344:101::2 producer_port = 36187
4182026-05-29T19:44:41.788ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 2 n_samples = 11 producer_id = f832abec-1084-4cbd-8043-186e70e862f7 producer_ip = fd00:1122:3344:101::2 producer_port = 36187
4192026-05-29T19:44:41.799ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
4202026-05-29T19:44:41.799ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 22fdeb8b-488b-4808-ad25-4b8f01f41850 producer_ip = fd00:1122:3344:101::1 producer_port = 37744
4212026-05-29T19:44:41.799ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
4222026-05-29T19:44:41.799ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 22fdeb8b-488b-4808-ad25-4b8f01f41850 producer_ip = fd00:1122:3344:101::1 producer_port = 37744
4232026-05-29T19:44:41.802ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 23 producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
4242026-05-29T19:44:41.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4252026-05-29T19:44:41.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4262026-05-29T19:44:41.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4272026-05-29T19:44:41.867ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 3 n_samples = 1624 producer_id = 22fdeb8b-488b-4808-ad25-4b8f01f41850 producer_ip = fd00:1122:3344:101::1 producer_port = 37744
4282026-05-29T19:44:41.870ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 1673
4292026-05-29T19:44:41.871ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 1673
4302026-05-29T19:44:41.871ZDEBGoximeter (clickhouse-client): unrolling 1673 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
4312026-05-29T19:44:41.919ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
4322026-05-29T19:44:41.939ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 2 table_name = fields_ipaddr
4332026-05-29T19:44:41.943ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 3298 table_name = fields_string
4342026-05-29T19:44:41.944ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 44 table_name = fields_u16
4352026-05-29T19:44:41.946ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 953 table_name = fields_u32
4362026-05-29T19:44:41.948ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1 table_name = fields_u64
4372026-05-29T19:44:41.950ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1869 table_name = fields_uuid
4382026-05-29T19:44:41.953ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1475 table_name = measurements_cumulativeu64
4392026-05-29T19:44:41.955ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 42 table_name = measurements_histogramu64
4402026-05-29T19:44:41.956ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 156 table_name = measurements_u64
4412026-05-29T19:44:41.970ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
4422026-05-29T19:44:42.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4432026-05-29T19:44:42.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4442026-05-29T19:44:42.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4452026-05-29T19:44:43.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4462026-05-29T19:44:43.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4472026-05-29T19:44:43.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4482026-05-29T19:44:44.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4492026-05-29T19:44:44.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4502026-05-29T19:44:44.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4512026-05-29T19:44:45.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4522026-05-29T19:44:45.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4532026-05-29T19:44:45.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4542026-05-29T19:44:46.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 56
4552026-05-29T19:44:46.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 56
4562026-05-29T19:44:46.546ZDEBGoximeter (clickhouse-client): unrolling 56 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
4572026-05-29T19:44:46.548ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
4582026-05-29T19:44:46.553ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 26 table_name = fields_string
4592026-05-29T19:44:46.554ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 28 table_name = fields_uuid
4602026-05-29T19:44:46.555ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 40 table_name = measurements_cumulativeu64
4612026-05-29T19:44:46.556ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 16 table_name = measurements_u64
4622026-05-29T19:44:46.647ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
4632026-05-29T19:44:46.791ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
4642026-05-29T19:44:46.791ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
4652026-05-29T19:44:46.802ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
4662026-05-29T19:44:46.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4672026-05-29T19:44:46.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4682026-05-29T19:44:46.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4692026-05-29T19:44:47.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4702026-05-29T19:44:47.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4712026-05-29T19:44:47.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4722026-05-29T19:44:48.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4732026-05-29T19:44:48.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4742026-05-29T19:44:48.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4752026-05-29T19:44:49.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4762026-05-29T19:44:49.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4772026-05-29T19:44:49.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4782026-05-29T19:44:50.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4792026-05-29T19:44:50.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4802026-05-29T19:44:50.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
4812026-05-29T19:44:51.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
4822026-05-29T19:44:51.546ZDEBGoximeter (clickhouse-client): unrolling 210 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
4832026-05-29T19:44:51.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 210
4842026-05-29T19:44:51.552ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
4852026-05-29T19:44:51.561ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 280 table_name = fields_i16
4862026-05-29T19:44:51.562ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 747 table_name = fields_string
4872026-05-29T19:44:51.564ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 140 table_name = fields_u16
4882026-05-29T19:44:51.565ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 145 table_name = fields_u32
4892026-05-29T19:44:51.566ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 144 table_name = fields_u8
4902026-05-29T19:44:51.568ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 448 table_name = fields_uuid
4912026-05-29T19:44:51.569ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 4 table_name = measurements_bool
4922026-05-29T19:44:51.570ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 155 table_name = measurements_cumulativeu64
4932026-05-29T19:44:51.571ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 1 table_name = measurements_f64
4942026-05-29T19:44:51.572ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_u64
4952026-05-29T19:44:51.647ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
4962026-05-29T19:44:51.780ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = f832abec-1084-4cbd-8043-186e70e862f7 producer_ip = fd00:1122:3344:101::2 producer_port = 36187
4972026-05-29T19:44:51.780ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = f832abec-1084-4cbd-8043-186e70e862f7 producer_ip = fd00:1122:3344:101::2 producer_port = 36187
4982026-05-29T19:44:51.782ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 2 n_samples = 11 producer_id = f832abec-1084-4cbd-8043-186e70e862f7 producer_ip = fd00:1122:3344:101::2 producer_port = 36187
4992026-05-29T19:44:51.799ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
5002026-05-29T19:44:51.799ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
5012026-05-29T19:44:51.802ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 24 producer_id = 1ae38df7-9909-4577-b497-a67c5cf30377 producer_ip = fd00:1122:3344:101::b producer_port = 43728
5022026-05-29T19:44:51.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5032026-05-29T19:44:51.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5042026-05-29T19:44:51.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5052026-05-29T19:44:52.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5062026-05-29T19:44:52.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5072026-05-29T19:44:52.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5082026-05-29T19:44:53.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5092026-05-29T19:44:53.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5102026-05-29T19:44:53.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5112026-05-29T19:44:54.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5122026-05-29T19:44:54.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5132026-05-29T19:44:54.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5142026-05-29T19:44:55.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5152026-05-29T19:44:55.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5162026-05-29T19:44:55.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5172026-05-29T19:44:56.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 105
5182026-05-29T19:44:56.546ZDEBGoximeter (database-inserter): inserting samples into database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_samples = 105
5192026-05-29T19:44:56.546ZDEBGoximeter (clickhouse-client): unrolling 105 total samples collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
5202026-05-29T19:44:56.549ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77
5212026-05-29T19:44:56.556ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 96 table_name = fields_string
5222026-05-29T19:44:56.557ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 35 table_name = fields_u16
5232026-05-29T19:44:56.558ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 63 table_name = fields_uuid
5242026-05-29T19:44:56.560ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 50 table_name = measurements_cumulativeu64
5252026-05-29T19:44:56.562ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 35 table_name = measurements_histogramu64
5262026-05-29T19:44:56.563ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d id = 73d9de17-7b01-4188-88a3-5b079937dd77 n_rows = 20 table_name = measurements_u64
5272026-05-29T19:44:56.646ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
5282026-05-29T19:44:56.770ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
5292026-05-29T19:44:56.770ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending
5302026-05-29T19:44:56.781ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "a79b5be9-aff2-47fd-ba68-61a04702a28d", "content-length": "1276", "date": "Fri, 29 May 2026 19:44:56 GMT"} })
5312026-05-29T19:44:56.781ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODMyYWJlYy0xMDg0LTRjYmQtODA0My0xODZlNzBlODYyZjcifX0%3D
5322026-05-29T19:44:56.784ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors/901ec26b-90dc-4813-8833-d7a056f32f59/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJmODMyYWJlYy0xMDg0LTRjYmQtODA0My0xODZlNzBlODYyZjcifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "2882c503-6dec-4b20-ad20-acbd07a1a092", "content-length": "29", "date": "Fri, 29 May 2026 19:44:56 GMT"} })
5332026-05-29T19:44:56.784ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::c]:35031 collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 30a1c5bf-58c5-4046-a5d9-3be1a7513740
5342026-05-29T19:44:56.784ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::a]:53953 collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 684ca080-099f-4581-bb2a-1cef504e74a7
5352026-05-29T19:44:56.784ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:8001 collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097
5362026-05-29T19:44:56.784ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 8 n_pruned_tasks = 0
5372026-05-29T19:44:56.785ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5382026-05-29T19:44:56.785ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 684ca080-099f-4581-bb2a-1cef504e74a7 producer_ip = fd00:1122:3344:101::a producer_port = 53953
5392026-05-29T19:44:56.785ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 30a1c5bf-58c5-4046-a5d9-3be1a7513740 producer_ip = fd00:1122:3344:101::c producer_port = 35031
5402026-05-29T19:44:56.785ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 30a1c5bf-58c5-4046-a5d9-3be1a7513740 producer_ip = fd00:1122:3344:101::c producer_port = 35031
5412026-05-29T19:44:56.785ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5422026-05-29T19:44:56.785ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5432026-05-29T19:44:56.785ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 30a1c5bf-58c5-4046-a5d9-3be1a7513740 producer_ip = fd00:1122:3344:101::c producer_port = 35031
5442026-05-29T19:44:56.785ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 684ca080-099f-4581-bb2a-1cef504e74a7 producer_ip = fd00:1122:3344:101::a producer_port = 53953
5452026-05-29T19:44:56.785ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 684ca080-099f-4581-bb2a-1cef504e74a7 producer_ip = fd00:1122:3344:101::a producer_port = 53953
5462026-05-29T19:44:56.789ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5472026-05-29T19:44:56.790ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5482026-05-29T19:44:56.790ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
5492026-05-29T19:44:56.790ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
5502026-05-29T19:44:56.791ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
5512026-05-29T19:44:56.792ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 5 producer_id = 684ca080-099f-4581-bb2a-1cef504e74a7 producer_ip = fd00:1122:3344:101::a producer_port = 53953
5522026-05-29T19:44:56.797ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 5 producer_id = 30a1c5bf-58c5-4046-a5d9-3be1a7513740 producer_ip = fd00:1122:3344:101::c producer_port = 35031
5532026-05-29T19:44:56.803ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = b551d0be-7606-4881-8553-e484015b2fa0 producer_ip = fd00:1122:3344:101::2 producer_port = 56326
5542026-05-29T19:44:56.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5552026-05-29T19:44:56.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5562026-05-29T19:44:56.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5572026-05-29T19:44:57.785ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5582026-05-29T19:44:57.785ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5592026-05-29T19:44:57.788ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5602026-05-29T19:44:57.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5612026-05-29T19:44:57.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5622026-05-29T19:44:57.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5632026-05-29T19:44:58.785ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5642026-05-29T19:44:58.785ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5652026-05-29T19:44:58.788ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5662026-05-29T19:44:58.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5672026-05-29T19:44:58.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5682026-05-29T19:44:58.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5692026-05-29T19:44:59.785ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5702026-05-29T19:44:59.785ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5712026-05-29T19:44:59.787ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5722026-05-29T19:44:59.807ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5732026-05-29T19:44:59.807ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5742026-05-29T19:44:59.809ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5752026-05-29T19:45:00.785ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5762026-05-29T19:45:00.785ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5772026-05-29T19:45:00.789ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = 6a22e29c-c9e0-4b65-99d6-9842f366c097 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5782026-05-29T19:45:00.808ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5792026-05-29T19:45:00.808ZDEBGoximeter (collection-task): collecting from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001
5802026-05-29T19:45:00.810ZDEBGoximeter (collection-task): collected results from producer collector_id = 901ec26b-90dc-4813-8833-d7a056f32f59 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 887d59d0-0b8c-4f71-81c3-e69fc6cbdfad producer_ip = fd00:1122:3344:101::2 producer_port = 8001