Max level shown:
1 [ Jul 21 19:13:55 Enabled. ]
2 [ Jul 21 19:13:55 Rereading configuration. ]
3 [ Jul 21 19:13:56 Rereading configuration. ]
4 [ Jul 21 19:14:00 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/oximeter-collector/bin/oximeter run /var/svc/manifest/site/oximeter/config.toml --address [fd00:1122:3344:101::d]:12223 --id c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc &"). ]
5 [ Jul 21 19:14:00 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72026-07-21T19:14:00.374ZDEBGoximeter: registered DTrace probes
82026-07-21T19:14:00.376ZINFOoximeter: starting oximeter server file = oximeter/collector/src/lib.rs:299
92026-07-21T19:14:00.376ZDEBGoximeter: creating ClickHouse client
102026-07-21T19:14:00.388ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
112026-07-21T19:14:07.188ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:888 id = db2a39d5-2206-4581-a196-7813f863f15e
122026-07-21T19:14:07.188ZDEBGoximeter (oximeter-agent): oximeter database does not exist, creating collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d
132026-07-21T19:14:07.188ZINFOoximeter (clickhouse-client): reading db version collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:788 id = db2a39d5-2206-4581-a196-7813f863f15e
142026-07-21T19:14:07.189ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:888 id = db2a39d5-2206-4581-a196-7813f863f15e
152026-07-21T19:14:07.189ZINFOoximeter (clickhouse-client): read oximeter database version collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:793 id = db2a39d5-2206-4581-a196-7813f863f15e version = 0
162026-07-21T19:14:07.189ZINFOoximeter (clickhouse-client): initializing oximeter schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:798 id = db2a39d5-2206-4581-a196-7813f863f15e
172026-07-21T19:14:07.189ZDEBGoximeter (clickhouse-client): initializing ClickHouse database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
182026-07-21T19:14:07.294ZINFOoximeter (clickhouse-client): inserting current version collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:819 id = db2a39d5-2206-4581-a196-7813f863f15e version = 14
192026-07-21T19:14:07.332ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = 6dfaef16-dd35-442c-83a0-8b5625e3bd67
202026-07-21T19:14:07.351ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /info versions = all
212026-07-21T19:14:07.352ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers versions = all
222026-07-21T19:14:07.352ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = DELETE path = /producers/{producer_id} versions = all
232026-07-21T19:14:07.352ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers/{producer_id} versions = all
242026-07-21T19:14:07.352ZINFOoximeter (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.1/src/server.rs:335 local_addr = [fd00:1122:3344:101::d]:12223
252026-07-21T19:14:07.361ZDEBGoximeter (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::d]:12223
262026-07-21T19:14:07.361ZDEBGoximeter: registered USDT probes
272026-07-21T19:14:07.361ZDEBGoximeter: contacting nexus
282026-07-21T19:14:07.412ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors
292026-07-21T19:14:07.412ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::a]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
302026-07-21T19:14:07.412ZWARNoximeter: failed to contact nexus, will retry in 249.879494ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:413
312026-07-21T19:14:07.664ZDEBGoximeter: contacting nexus
322026-07-21T19:14:07.664ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors
332026-07-21T19:14:07.664ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::a]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
342026-07-21T19:14:07.664ZWARNoximeter: failed to contact nexus, will retry in 543.684303ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:413
352026-07-21T19:14:08.209ZDEBGoximeter: contacting nexus
362026-07-21T19:14:08.209ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors
372026-07-21T19:14:08.209ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::b]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
382026-07-21T19:14:08.209ZWARNoximeter: failed to contact nexus, will retry in 1.270858798s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:413
392026-07-21T19:14:09.480ZDEBGoximeter: contacting nexus
402026-07-21T19:14:09.480ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors
412026-07-21T19:14:09.481ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::a]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
422026-07-21T19:14:09.481ZWARNoximeter: failed to contact nexus, will retry in 2.059076943s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:413
432026-07-21T19:14:11.540ZDEBGoximeter: contacting nexus
442026-07-21T19:14:11.540ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors
452026-07-21T19:14:11.608ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors", status: 204, headers: {"x-request-id": "4be1a9fd-3e9c-4574-a8a2-9412cca8fb8f", "date": "Tue, 21 Jul 2026 19:14:11 GMT"} })
462026-07-21T19:14:11.611ZINFOoximeter: oximeter registered with nexus file = oximeter/collector/src/lib.rs:431 id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc
472026-07-21T19:14:11.611ZINFOoximeter (oximeter-agent): starting refresh list task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:626
482026-07-21T19:14:11.612ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:629
492026-07-21T19:14:11.617ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending
502026-07-21T19:14:11.617ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::a]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
512026-07-21T19:14:11.617ZERROoximeter (oximeter-agent): error fetching next assigned producer, abandoning this refresh attempt collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending): error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending): client error (Connect): tcp connect error: Connection refused (os error 146) file = oximeter/collector/src/agent.rs:667
522026-07-21T19:14:12.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 4
532026-07-21T19:14:12.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 4
542026-07-21T19:14:12.333ZDEBGoximeter (clickhouse-client): unrolling 4 total samples collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
552026-07-21T19:14:12.335ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
562026-07-21T19:14:12.337ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:database_queue_depth"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "collector_sink", field_type: String, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:12.335856266Z } timeseries_name = oximeter_collector:database_queue_depth
572026-07-21T19:14:12.337ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:database_samples_dropped"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "collector_sink", field_type: String, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:12.335851376Z } timeseries_name = oximeter_collector:database_samples_dropped
582026-07-21T19:14:12.337ZDEBGoximeter (clickhouse-client): inserting 2 new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
592026-07-21T19:14:12.342ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 4 table_name = fields_ipaddr
602026-07-21T19:14:12.344ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 4 table_name = fields_string
612026-07-21T19:14:12.346ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 4 table_name = fields_u16
622026-07-21T19:14:12.349ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 4 table_name = fields_uuid
632026-07-21T19:14:12.351ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 2 table_name = measurements_cumulativeu64
642026-07-21T19:14:12.354ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 2 table_name = measurements_histogramu64
652026-07-21T19:14:12.434ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:276
662026-07-21T19:14:17.333ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d
672026-07-21T19:14:17.333ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d
682026-07-21T19:14:22.334ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d
692026-07-21T19:14:22.334ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d
702026-07-21T19:14:26.611ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:629
712026-07-21T19:14:26.611ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending
722026-07-21T19:14:26.618ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "5270720b-4fa0-4870-8f0a-0a4b726be034", "content-length": "298", "date": "Tue, 21 Jul 2026 19:14:26 GMT"} })
732026-07-21T19:14:26.620ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiYmM5Y2U2Ni01ODc5LTQ1MGUtYjM3Yy04ZWRhNjE1OWRmYWEifX0%3D
742026-07-21T19:14:26.631ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiYmM5Y2U2Ni01ODc5LTQ1MGUtYjM3Yy04ZWRhNjE1OWRmYWEifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "fb43dd36-59e2-40c5-a8e4-1dbfb5d349af", "content-length": "29", "date": "Tue, 21 Jul 2026 19:14:26 GMT"} })
752026-07-21T19:14:26.631ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:8001 collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa
762026-07-21T19:14:26.631ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:704 n_current_tasks = 1 n_pruned_tasks = 0
772026-07-21T19:14:26.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
782026-07-21T19:14:26.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
792026-07-21T19:14:26.701ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
802026-07-21T19:14:27.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 26
812026-07-21T19:14:27.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 26
822026-07-21T19:14:27.334ZDEBGoximeter (clickhouse-client): unrolling 26 total samples collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
832026-07-21T19:14:27.334ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
842026-07-21T19:14:27.346ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333904467Z } timeseries_name = ddm_router:originated_tunnel_endpoints
852026-07-21T19:14:27.346ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333901217Z } timeseries_name = ddm_router:originated_underlay_prefixes
862026-07-21T19:14:27.346ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333912217Z } timeseries_name = ddm_session:advertisements_received
872026-07-21T19:14:27.346ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333910547Z } timeseries_name = ddm_session:advertisements_sent
882026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333925917Z } timeseries_name = ddm_session:imported_tunnel_endpoints
892026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333924348Z } timeseries_name = ddm_session:imported_underlay_prefixes
902026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_address_changes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333915717Z } timeseries_name = ddm_session:peer_address_changes
912026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_expirations"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333914197Z } timeseries_name = ddm_session:peer_expirations
922026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_sessions_established"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333917517Z } timeseries_name = ddm_session:peer_sessions_established
932026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333908647Z } timeseries_name = ddm_session:solicitations_received
942026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333906627Z } timeseries_name = ddm_session:solicitations_sent
952026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:update_send_fail"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333921868Z } timeseries_name = ddm_session:update_send_fail
962026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333920637Z } timeseries_name = ddm_session:updates_received
972026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:27.333919227Z } timeseries_name = ddm_session:updates_sent
982026-07-21T19:14:27.347ZDEBGoximeter (clickhouse-client): inserting 14 new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
992026-07-21T19:14:27.351ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 50 table_name = fields_string
1002026-07-21T19:14:27.352ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 52 table_name = fields_uuid
1012026-07-21T19:14:27.354ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 20 table_name = measurements_cumulativeu64
1022026-07-21T19:14:27.356ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 6 table_name = measurements_u64
1032026-07-21T19:14:27.434ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:276
1042026-07-21T19:14:27.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1052026-07-21T19:14:27.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1062026-07-21T19:14:27.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1072026-07-21T19:14:28.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1082026-07-21T19:14:28.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1092026-07-21T19:14:28.688ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1102026-07-21T19:14:29.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1112026-07-21T19:14:29.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1122026-07-21T19:14:29.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1132026-07-21T19:14:30.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1142026-07-21T19:14:30.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1152026-07-21T19:14:30.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1162026-07-21T19:14:31.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1172026-07-21T19:14:31.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1182026-07-21T19:14:31.690ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1192026-07-21T19:14:32.334ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 130
1202026-07-21T19:14:32.334ZDEBGoximeter (clickhouse-client): unrolling 130 total samples collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
1212026-07-21T19:14:32.334ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 130
1222026-07-21T19:14:32.334ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
1232026-07-21T19:14:32.357ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 50 table_name = fields_string
1242026-07-21T19:14:32.359ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 52 table_name = fields_uuid
1252026-07-21T19:14:32.361ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 100 table_name = measurements_cumulativeu64
1262026-07-21T19:14:32.362ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 30 table_name = measurements_u64
1272026-07-21T19:14:32.435ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:276
1282026-07-21T19:14:32.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1292026-07-21T19:14:32.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1302026-07-21T19:14:32.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1312026-07-21T19:14:33.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1322026-07-21T19:14:33.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1332026-07-21T19:14:33.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1342026-07-21T19:14:34.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1352026-07-21T19:14:34.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1362026-07-21T19:14:34.690ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1372026-07-21T19:14:35.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1382026-07-21T19:14:35.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1392026-07-21T19:14:35.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1402026-07-21T19:14:36.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1412026-07-21T19:14:36.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1422026-07-21T19:14:36.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1432026-07-21T19:14:37.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 130
1442026-07-21T19:14:37.334ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 130
1452026-07-21T19:14:37.334ZDEBGoximeter (clickhouse-client): unrolling 130 total samples collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
1462026-07-21T19:14:37.334ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
1472026-07-21T19:14:37.339ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 50 table_name = fields_string
1482026-07-21T19:14:37.341ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 52 table_name = fields_uuid
1492026-07-21T19:14:37.342ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 100 table_name = measurements_cumulativeu64
1502026-07-21T19:14:37.344ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 30 table_name = measurements_u64
1512026-07-21T19:14:37.434ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:276
1522026-07-21T19:14:37.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1532026-07-21T19:14:37.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1542026-07-21T19:14:37.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1552026-07-21T19:14:38.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1562026-07-21T19:14:38.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1572026-07-21T19:14:38.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1582026-07-21T19:14:39.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1592026-07-21T19:14:39.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1602026-07-21T19:14:39.690ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1612026-07-21T19:14:40.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1622026-07-21T19:14:40.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1632026-07-21T19:14:40.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1642026-07-21T19:14:41.611ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:629
1652026-07-21T19:14:41.611ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending
1662026-07-21T19:14:41.621ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "f93ec594-e791-4855-990d-c6dc90fcd75d", "content-length": "583", "date": "Tue, 21 Jul 2026 19:14:41 GMT"} })
1672026-07-21T19:14:41.621ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiYmM5Y2U2Ni01ODc5LTQ1MGUtYjM3Yy04ZWRhNjE1OWRmYWEifX0%3D
1682026-07-21T19:14:41.625ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJiYmM5Y2U2Ni01ODc5LTQ1MGUtYjM3Yy04ZWRhNjE1OWRmYWEifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "ac37cc4b-131d-44f1-8d94-82da8c419e33", "content-length": "29", "date": "Tue, 21 Jul 2026 19:14:41 GMT"} })
1692026-07-21T19:14:41.625ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:40979 collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 16870ef3-3e24-4393-8a2a-d71808ed1392
1702026-07-21T19:14:41.625ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:8001 collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69
1712026-07-21T19:14:41.625ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:704 n_current_tasks = 3 n_pruned_tasks = 0
1722026-07-21T19:14:41.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1732026-07-21T19:14:41.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1742026-07-21T19:14:41.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1752026-07-21T19:14:41.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1762026-07-21T19:14:41.946ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1772026-07-21T19:14:41.955ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1782026-07-21T19:14:42.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 144
1792026-07-21T19:14:42.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 144
1802026-07-21T19:14:42.333ZDEBGoximeter (clickhouse-client): unrolling 144 total samples collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
1812026-07-21T19:14:42.333ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
1822026-07-21T19:14:42.338ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 76 table_name = fields_string
1832026-07-21T19:14:42.340ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 80 table_name = fields_uuid
1842026-07-21T19:14:42.341ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 110 table_name = measurements_cumulativeu64
1852026-07-21T19:14:42.343ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 34 table_name = measurements_u64
1862026-07-21T19:14:42.434ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:276
1872026-07-21T19:14:42.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1882026-07-21T19:14:42.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1892026-07-21T19:14:42.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1902026-07-21T19:14:42.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1912026-07-21T19:14:42.945ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1922026-07-21T19:14:42.946ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1932026-07-21T19:14:43.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1942026-07-21T19:14:43.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1952026-07-21T19:14:43.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1962026-07-21T19:14:43.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1972026-07-21T19:14:43.945ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1982026-07-21T19:14:43.946ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1992026-07-21T19:14:44.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2002026-07-21T19:14:44.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2012026-07-21T19:14:44.690ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2022026-07-21T19:14:44.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2032026-07-21T19:14:44.945ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2042026-07-21T19:14:44.947ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2052026-07-21T19:14:45.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2062026-07-21T19:14:45.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2072026-07-21T19:14:45.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2082026-07-21T19:14:45.946ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2092026-07-21T19:14:45.946ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2102026-07-21T19:14:45.947ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2112026-07-21T19:14:46.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2122026-07-21T19:14:46.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2132026-07-21T19:14:46.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2142026-07-21T19:14:46.946ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2152026-07-21T19:14:46.947ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2162026-07-21T19:14:46.947ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2172026-07-21T19:14:47.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 200
2182026-07-21T19:14:47.334ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 200
2192026-07-21T19:14:47.334ZDEBGoximeter (clickhouse-client): unrolling 200 total samples collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
2202026-07-21T19:14:47.334ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
2212026-07-21T19:14:47.338ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 76 table_name = fields_string
2222026-07-21T19:14:47.340ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 80 table_name = fields_uuid
2232026-07-21T19:14:47.342ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 150 table_name = measurements_cumulativeu64
2242026-07-21T19:14:47.344ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 50 table_name = measurements_u64
2252026-07-21T19:14:47.435ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:276
2262026-07-21T19:14:47.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2272026-07-21T19:14:47.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2282026-07-21T19:14:47.690ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2292026-07-21T19:14:47.946ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2302026-07-21T19:14:47.947ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2312026-07-21T19:14:47.948ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2322026-07-21T19:14:48.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2332026-07-21T19:14:48.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2342026-07-21T19:14:48.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2352026-07-21T19:14:48.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2362026-07-21T19:14:48.946ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2372026-07-21T19:14:48.949ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2382026-07-21T19:14:49.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2392026-07-21T19:14:49.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2402026-07-21T19:14:49.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2412026-07-21T19:14:49.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2422026-07-21T19:14:49.945ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2432026-07-21T19:14:49.946ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2442026-07-21T19:14:50.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2452026-07-21T19:14:50.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2462026-07-21T19:14:50.690ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2472026-07-21T19:14:50.816ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 16870ef3-3e24-4393-8a2a-d71808ed1392 producer_ip = fd00:1122:3344:101::2 producer_port = 40979
2482026-07-21T19:14:50.816ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 16870ef3-3e24-4393-8a2a-d71808ed1392 producer_ip = fd00:1122:3344:101::2 producer_port = 40979
2492026-07-21T19:14:50.826ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 2 n_samples = 11 producer_id = 16870ef3-3e24-4393-8a2a-d71808ed1392 producer_ip = fd00:1122:3344:101::2 producer_port = 40979
2502026-07-21T19:14:50.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2512026-07-21T19:14:50.946ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2522026-07-21T19:14:50.946ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2532026-07-21T19:14:51.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2542026-07-21T19:14:51.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2552026-07-21T19:14:51.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2562026-07-21T19:14:51.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2572026-07-21T19:14:51.945ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2582026-07-21T19:14:51.946ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2592026-07-21T19:14:52.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 211
2602026-07-21T19:14:52.333ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 211
2612026-07-21T19:14:52.333ZDEBGoximeter (clickhouse-client): unrolling 211 total samples collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
2622026-07-21T19:14:52.333ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
2632026-07-21T19:14:52.336ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:52.333935626Z } timeseries_name = http_service:request_latency_histogram
2642026-07-21T19:14:52.337ZDEBGoximeter (clickhouse-client): inserting 1 new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
2652026-07-21T19:14:52.341ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 98 table_name = fields_string
2662026-07-21T19:14:52.343ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 11 table_name = fields_u16
2672026-07-21T19:14:52.345ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 91 table_name = fields_uuid
2682026-07-21T19:14:52.346ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 150 table_name = measurements_cumulativeu64
2692026-07-21T19:14:52.352ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 11 table_name = measurements_histogramu64
2702026-07-21T19:14:52.354ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 50 table_name = measurements_u64
2712026-07-21T19:14:52.434ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:276
2722026-07-21T19:14:52.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2732026-07-21T19:14:52.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2742026-07-21T19:14:52.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2752026-07-21T19:14:52.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2762026-07-21T19:14:52.945ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2772026-07-21T19:14:52.946ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2782026-07-21T19:14:53.463ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2792026-07-21T19:14:53.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2802026-07-21T19:14:53.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2812026-07-21T19:14:53.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2822026-07-21T19:14:53.946ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2832026-07-21T19:14:53.946ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2842026-07-21T19:14:53.947ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2852026-07-21T19:14:54.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2862026-07-21T19:14:54.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2872026-07-21T19:14:54.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2882026-07-21T19:14:54.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2892026-07-21T19:14:54.945ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2902026-07-21T19:14:54.946ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2912026-07-21T19:14:55.688ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2922026-07-21T19:14:55.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2932026-07-21T19:14:55.690ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2942026-07-21T19:14:55.946ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2952026-07-21T19:14:55.946ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2962026-07-21T19:14:55.947ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2972026-07-21T19:14:56.612ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:629
2982026-07-21T19:14:56.612ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending
2992026-07-21T19:14:56.619ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "a425fdde-16eb-42f2-a828-45c8477a2bf8", "content-length": "995", "date": "Tue, 21 Jul 2026 19:14:56 GMT"} })
3002026-07-21T19:14:56.619ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMTY1YmZjNi04Y2NjLTRmODEtYTA2Ni0wNjVjZDNmNTZhODEifX0%3D
3012026-07-21T19:14:56.622ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors/c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJjMTY1YmZjNi04Y2NjLTRmODEtYTA2Ni0wNjVjZDNmNTZhODEifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "e74d3a61-4533-45b3-b888-8ccb1cd2baae", "content-length": "29", "date": "Tue, 21 Jul 2026 19:14:56 GMT"} })
3022026-07-21T19:14:56.622ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::a]:44821 collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 315b75e8-783a-4ab0-a3b6-7ad1bd507336
3032026-07-21T19:14:56.622ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:36965 collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 9ce44447-fa52-40d6-9ce7-0219bde8959a
3042026-07-21T19:14:56.622ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:4677 collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81
3052026-07-21T19:14:56.622ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:704 n_current_tasks = 6 n_pruned_tasks = 0
3062026-07-21T19:14:56.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3072026-07-21T19:14:56.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3082026-07-21T19:14:56.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3092026-07-21T19:14:56.946ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3102026-07-21T19:14:56.946ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3112026-07-21T19:14:56.948ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3122026-07-21T19:14:57.334ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 202
3132026-07-21T19:14:57.334ZDEBGoximeter (database-inserter): inserting samples into database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_samples = 202
3142026-07-21T19:14:57.334ZDEBGoximeter (clickhouse-client): unrolling 202 total samples collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
3152026-07-21T19:14:57.334ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
3162026-07-21T19:14:57.337ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:collections"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "base_route", field_type: String, source: Metric, description: "" }, FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "producer_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "producer_ip", field_type: IpAddr, source: Metric, description: "" }, FieldSchema { name: "producer_port", field_type: U16, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:57.334288587Z } timeseries_name = oximeter_collector:collections
3172026-07-21T19:14:57.337ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:samples_collected"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "base_route", field_type: String, source: Metric, description: "" }, FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "producer_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "producer_ip", field_type: IpAddr, source: Metric, description: "" }, FieldSchema { name: "producer_port", field_type: U16, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-07-21T19:14:57.334292267Z } timeseries_name = oximeter_collector:samples_collected
3182026-07-21T19:14:57.338ZDEBGoximeter (clickhouse-client): inserting 2 new timeseries schema collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e
3192026-07-21T19:14:57.340ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 4 table_name = fields_ipaddr
3202026-07-21T19:14:57.342ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 78 table_name = fields_string
3212026-07-21T19:14:57.343ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 4 table_name = fields_u16
3222026-07-21T19:14:57.344ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 84 table_name = fields_uuid
3232026-07-21T19:14:57.345ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 152 table_name = measurements_cumulativeu64
3242026-07-21T19:14:57.346ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d id = db2a39d5-2206-4581-a196-7813f863f15e n_rows = 50 table_name = measurements_u64
3252026-07-21T19:14:57.435ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:276
3262026-07-21T19:14:57.504ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3272026-07-21T19:14:57.504ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3282026-07-21T19:14:57.512ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3292026-07-21T19:14:57.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3302026-07-21T19:14:57.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3312026-07-21T19:14:57.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3322026-07-21T19:14:57.946ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3332026-07-21T19:14:57.946ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3342026-07-21T19:14:57.947ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3352026-07-21T19:14:58.167ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 315b75e8-783a-4ab0-a3b6-7ad1bd507336 producer_ip = fd00:1122:3344:101::a producer_port = 44821
3362026-07-21T19:14:58.167ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 315b75e8-783a-4ab0-a3b6-7ad1bd507336 producer_ip = fd00:1122:3344:101::a producer_port = 44821
3372026-07-21T19:14:58.168ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 6 producer_id = 315b75e8-783a-4ab0-a3b6-7ad1bd507336 producer_ip = fd00:1122:3344:101::a producer_port = 44821
3382026-07-21T19:14:58.504ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3392026-07-21T19:14:58.504ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3402026-07-21T19:14:58.505ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3412026-07-21T19:14:58.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3422026-07-21T19:14:58.688ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3432026-07-21T19:14:58.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3442026-07-21T19:14:58.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3452026-07-21T19:14:58.945ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3462026-07-21T19:14:58.946ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3472026-07-21T19:14:59.503ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3482026-07-21T19:14:59.504ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3492026-07-21T19:14:59.504ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3502026-07-21T19:14:59.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3512026-07-21T19:14:59.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3522026-07-21T19:14:59.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3532026-07-21T19:14:59.946ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3542026-07-21T19:14:59.946ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3552026-07-21T19:14:59.947ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3562026-07-21T19:15:00.504ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3572026-07-21T19:15:00.504ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3582026-07-21T19:15:00.504ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = c165bfc6-8ccc-4f81-a066-065cd3f56a81 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3592026-07-21T19:15:00.687ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3602026-07-21T19:15:00.687ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3612026-07-21T19:15:00.689ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = bbc9ce66-5879-450e-b37c-8eda6159dfaa producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3622026-07-21T19:15:00.816ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 16870ef3-3e24-4393-8a2a-d71808ed1392 producer_ip = fd00:1122:3344:101::2 producer_port = 40979
3632026-07-21T19:15:00.816ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 16870ef3-3e24-4393-8a2a-d71808ed1392 producer_ip = fd00:1122:3344:101::2 producer_port = 40979
3642026-07-21T19:15:00.818ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 2 n_samples = 11 producer_id = 16870ef3-3e24-4393-8a2a-d71808ed1392 producer_ip = fd00:1122:3344:101::2 producer_port = 40979
3652026-07-21T19:15:00.945ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3662026-07-21T19:15:00.945ZDEBGoximeter (collection-task): collecting from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
3672026-07-21T19:15:00.947ZDEBGoximeter (collection-task): collected results from producer collector_id = c6fd564c-2f4d-4c6f-a355-d5d7bf5718fc collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 8fe5a0d7-3374-4862-9184-39628662ba69 producer_ip = fd00:1122:3344:101::2 producer_port = 8001