Max level shown:
1 [ Jun 9 02:20:45 Enabled. ]
2 [ Jun 9 02:20:45 Rereading configuration. ]
3 [ Jun 9 02:20:46 Rereading configuration. ]
4 [ Jun 9 02:20:50 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/oximeter-collector/bin/oximeter run /var/svc/manifest/site/oximeter/config.toml --address [fd00:1122:3344:101::d]:12223 --id e341817b-fe8a-48ae-9a66-66ec8c47e844 &"). ]
5 [ Jun 9 02:20:50 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72026-06-09T02:20:50.704ZDEBGoximeter: registered DTrace probes
82026-06-09T02:20:50.708ZINFOoximeter: starting oximeter server file = oximeter/collector/src/lib.rs:301
92026-06-09T02:20:50.709ZDEBGoximeter: creating ClickHouse client
102026-06-09T02:20:50.746ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
112026-06-09T02:20:54.030ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:890 id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
122026-06-09T02:20:54.030ZDEBGoximeter (oximeter-agent): oximeter database does not exist, creating collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d
132026-06-09T02:20:54.030ZINFOoximeter (clickhouse-client): reading db version collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:790 id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
142026-06-09T02:20:54.031ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:890 id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
152026-06-09T02:20:54.031ZINFOoximeter (clickhouse-client): read oximeter database version collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:795 id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 version = 0
162026-06-09T02:20:54.031ZINFOoximeter (clickhouse-client): initializing oximeter schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:800 id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
172026-06-09T02:20:54.031ZDEBGoximeter (clickhouse-client): initializing ClickHouse database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
182026-06-09T02:20:54.577ZINFOoximeter (clickhouse-client): inserting current version collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:821 id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 version = 14
192026-06-09T02:20:54.609ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 1c16b340-0284-4fa3-8ee9-58b7785df15c
202026-06-09T02:20:54.651ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /info versions = all
212026-06-09T02:20:54.651ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers versions = all
222026-06-09T02:20:54.651ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = DELETE path = /producers/{producer_id} versions = all
232026-06-09T02:20:54.651ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers/{producer_id} versions = all
242026-06-09T02:20:54.651ZINFOoximeter (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.17.0/src/server.rs:335 local_addr = [fd00:1122:3344:101::d]:12223
252026-06-09T02:20:54.656ZDEBGoximeter (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::d]:12223
262026-06-09T02:20:54.656ZDEBGoximeter: registered USDT probes
272026-06-09T02:20:54.656ZDEBGoximeter: contacting nexus
282026-06-09T02:20:54.693ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors
292026-06-09T02:20:54.693ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::a]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
302026-06-09T02:20:54.693ZWARNoximeter: failed to contact nexus, will retry in 163.580127ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:446
312026-06-09T02:20:54.858ZDEBGoximeter: contacting nexus
322026-06-09T02:20:54.858ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors
332026-06-09T02:20:54.858ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::c]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
342026-06-09T02:20:54.858ZWARNoximeter: failed to contact nexus, will retry in 626.605473ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:446
352026-06-09T02:20:55.486ZDEBGoximeter: contacting nexus
362026-06-09T02:20:55.486ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors
372026-06-09T02:20:55.487ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::c]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
382026-06-09T02:20:55.487ZWARNoximeter: failed to contact nexus, will retry in 1.191205588s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:446
392026-06-09T02:20:56.679ZDEBGoximeter: contacting nexus
402026-06-09T02:20:56.679ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors
412026-06-09T02:20:56.801ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors", status: 204, headers: {"x-request-id": "f89af04d-f2c4-43f7-96d0-cc0b8bded683", "date": "Tue, 09 Jun 2026 02:20:56 GMT"} })
422026-06-09T02:20:56.801ZINFOoximeter: oximeter registered with nexus file = oximeter/collector/src/lib.rs:464 id = e341817b-fe8a-48ae-9a66-66ec8c47e844
432026-06-09T02:20:56.801ZINFOoximeter (oximeter-agent): starting refresh list task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:562
442026-06-09T02:20:56.802ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
452026-06-09T02:20:56.810ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending
462026-06-09T02:20:56.956ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "0545c8a6-2e95-4376-8adc-c5afc20acc31", "content-length": "29", "date": "Tue, 09 Jun 2026 02:20:56 GMT"} })
472026-06-09T02:20:56.956ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 0 n_pruned_tasks = 0
482026-06-09T02:20:59.617ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d
492026-06-09T02:20:59.622ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d
502026-06-09T02:21:04.617ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d
512026-06-09T02:21:04.622ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d
522026-06-09T02:21:09.617ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d
532026-06-09T02:21:09.622ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d
542026-06-09T02:21:11.803ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
552026-06-09T02:21:11.803ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending
562026-06-09T02:21:11.845ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "de9f0fcd-2609-42af-bbed-9abc8f42675b", "content-length": "300", "date": "Tue, 09 Jun 2026 02:21:11 GMT"} })
572026-06-09T02:21:11.845ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI2MzJiZWRhNC04ZjIxLTQyN2YtYTQyMi1lZWUxZGE4YjZiMDQifX0%3D
582026-06-09T02:21:11.890ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiI2MzJiZWRhNC04ZjIxLTQyN2YtYTQyMi1lZWUxZGE4YjZiMDQifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "4de2f0a7-8c1e-4237-8240-a6a26bbc83b3", "content-length": "29", "date": "Tue, 09 Jun 2026 02:21:11 GMT"} })
592026-06-09T02:21:11.890ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:52706 collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04
602026-06-09T02:21:11.890ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 1 n_pruned_tasks = 0
612026-06-09T02:21:11.891ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
622026-06-09T02:21:11.900ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
632026-06-09T02:21:11.900ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
642026-06-09T02:21:11.923ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
652026-06-09T02:21:14.617ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 141
662026-06-09T02:21:14.617ZDEBGoximeter (clickhouse-client): unrolling 141 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
672026-06-09T02:21:14.622ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 141
682026-06-09T02:21:14.637ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
692026-06-09T02:21:14.639ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("dendrite:sample_collection_duration"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Metric, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Metric, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Metric, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Metric, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Metric, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Metric, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Metric, description: "" }}, datum_type: F64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.637872475Z } timeseries_name = dendrite:sample_collection_duration
702026-06-09T02:21:14.639ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:collections"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "base_route", field_type: String, source: Metric, description: "" }, FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "producer_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "producer_ip", field_type: IpAddr, source: Metric, description: "" }, FieldSchema { name: "producer_port", field_type: U16, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633001830Z } timeseries_name = oximeter_collector:collections
712026-06-09T02:21:14.639ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_data_link:link_enabled"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_id", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "port_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: Bool, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633126063Z } timeseries_name = switch_data_link:link_enabled
722026-06-09T02:21:14.639ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_data_link:link_up"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_id", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "port_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: Bool, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633247046Z } timeseries_name = switch_data_link:link_up
732026-06-09T02:21:14.640ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:capacity"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633387929Z } timeseries_name = switch_table:capacity
742026-06-09T02:21:14.640ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:collisions"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633746718Z } timeseries_name = switch_table:collisions
752026-06-09T02:21:14.640ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:delete_misses"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633881881Z } timeseries_name = switch_table:delete_misses
762026-06-09T02:21:14.640ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:deletes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633604774Z } timeseries_name = switch_table:deletes
772026-06-09T02:21:14.640ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:exhaustion"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633955093Z } timeseries_name = switch_table:exhaustion
782026-06-09T02:21:14.640ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:inserts"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633527163Z } timeseries_name = switch_table:inserts
792026-06-09T02:21:14.640ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:occupancy"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633454751Z } timeseries_name = switch_table:occupancy
802026-06-09T02:21:14.640ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:update_misses"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633814759Z } timeseries_name = switch_table:update_misses
812026-06-09T02:21:14.641ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:updates"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:14.633678536Z } timeseries_name = switch_table:updates
822026-06-09T02:21:14.644ZDEBGoximeter (clickhouse-client): inserting 13 new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
832026-06-09T02:21:14.651ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 280 table_name = fields_i16
842026-06-09T02:21:14.653ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = fields_ipaddr
852026-06-09T02:21:14.655ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 722 table_name = fields_string
862026-06-09T02:21:14.657ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 142 table_name = fields_u16
872026-06-09T02:21:14.660ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 145 table_name = fields_u32
882026-06-09T02:21:14.663ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 144 table_name = fields_u8
892026-06-09T02:21:14.665ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 422 table_name = fields_uuid
902026-06-09T02:21:14.667ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 4 table_name = measurements_bool
912026-06-09T02:21:14.669ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 106 table_name = measurements_cumulativeu64
922026-06-09T02:21:14.671ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 1 table_name = measurements_f64
932026-06-09T02:21:14.673ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 30 table_name = measurements_u64
942026-06-09T02:21:14.725ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
952026-06-09T02:21:19.617ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d
962026-06-09T02:21:19.622ZDEBGoximeter (database-inserter): batch interval expired, but no samples to insert collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d
972026-06-09T02:21:21.901ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
982026-06-09T02:21:21.901ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
992026-06-09T02:21:21.919ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
1002026-06-09T02:21:24.617ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 140
1012026-06-09T02:21:24.617ZDEBGoximeter (clickhouse-client): unrolling 140 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
1022026-06-09T02:21:24.622ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
1032026-06-09T02:21:24.622ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 140
1042026-06-09T02:21:24.664ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 280 table_name = fields_i16
1052026-06-09T02:21:24.666ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 721 table_name = fields_string
1062026-06-09T02:21:24.668ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 140 table_name = fields_u16
1072026-06-09T02:21:24.670ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 145 table_name = fields_u32
1082026-06-09T02:21:24.671ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 144 table_name = fields_u8
1092026-06-09T02:21:24.673ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 420 table_name = fields_uuid
1102026-06-09T02:21:24.675ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 4 table_name = measurements_bool
1112026-06-09T02:21:24.677ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 105 table_name = measurements_cumulativeu64
1122026-06-09T02:21:24.678ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 1 table_name = measurements_f64
1132026-06-09T02:21:24.680ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 30 table_name = measurements_u64
1142026-06-09T02:21:24.723ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
1152026-06-09T02:21:26.803ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
1162026-06-09T02:21:26.803ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending
1172026-06-09T02:21:26.811ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "daa8326a-f56f-4438-8fd6-b287129d53a3", "content-length": "574", "date": "Tue, 09 Jun 2026 02:21:26 GMT"} })
1182026-06-09T02:21:26.811ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D
1192026-06-09T02:21:26.814ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "1e16c595-768a-4b99-adfb-4e43de962550", "content-length": "29", "date": "Tue, 09 Jun 2026 02:21:26 GMT"} })
1202026-06-09T02:21:26.814ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::c]:43239 collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1
1212026-06-09T02:21:26.814ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:8001 collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7
1222026-06-09T02:21:26.814ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 3 n_pruned_tasks = 0
1232026-06-09T02:21:26.815ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1242026-06-09T02:21:26.815ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
1252026-06-09T02:21:26.815ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
1262026-06-09T02:21:26.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
1272026-06-09T02:21:26.815ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1282026-06-09T02:21:26.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1292026-06-09T02:21:26.820ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1302026-06-09T02:21:26.825ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 9 producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
1312026-06-09T02:21:27.814ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1322026-06-09T02:21:27.814ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1332026-06-09T02:21:27.817ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1342026-06-09T02:21:28.815ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1352026-06-09T02:21:28.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1362026-06-09T02:21:28.818ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1372026-06-09T02:21:29.617ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 89
1382026-06-09T02:21:29.617ZDEBGoximeter (clickhouse-client): unrolling 89 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
1392026-06-09T02:21:29.621ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
1402026-06-09T02:21:29.622ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 89
1412026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:cpus_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.619175762Z } timeseries_name = collection_target:cpus_provisioned
1422026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:ram_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.619320695Z } timeseries_name = collection_target:ram_provisioned
1432026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("collection_target:virtual_disk_space_provisioned"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }}, datum_type: I64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.619412647Z } timeseries_name = collection_target:virtual_disk_space_provisioned
1442026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("database_transaction:retry_data"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "attempt", field_type: U32, source: Metric, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }}, datum_type: F64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.619612062Z } timeseries_name = database_transaction:retry_data
1452026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.617715528Z } timeseries_name = ddm_router:originated_tunnel_endpoints
1462026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.617636176Z } timeseries_name = ddm_router:originated_underlay_prefixes
1472026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.618022715Z } timeseries_name = ddm_session:advertisements_received
1482026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.617946563Z } timeseries_name = ddm_session:advertisements_sent
1492026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.618628479Z } timeseries_name = ddm_session:imported_tunnel_endpoints
1502026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.618553117Z } timeseries_name = ddm_session:imported_underlay_prefixes
1512026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_address_changes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.618174058Z } timeseries_name = ddm_session:peer_address_changes
1522026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_expirations"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.618098647Z } timeseries_name = ddm_session:peer_expirations
1532026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_sessions_established"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.618249520Z } timeseries_name = ddm_session:peer_sessions_established
1542026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.617870561Z } timeseries_name = ddm_session:solicitations_received
1552026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.617793329Z } timeseries_name = ddm_session:solicitations_sent
1562026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:update_send_fail"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.618477176Z } timeseries_name = ddm_session:update_send_fail
1572026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.618400444Z } timeseries_name = ddm_session:updates_received
1582026-06-09T02:21:29.623ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.618325102Z } timeseries_name = ddm_session:updates_sent
1592026-06-09T02:21:29.624ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:29.619062409Z } timeseries_name = http_service:request_latency_histogram
1602026-06-09T02:21:29.624ZDEBGoximeter (clickhouse-client): inserting 19 new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
1612026-06-09T02:21:29.627ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 4 table_name = fields_ipaddr
1622026-06-09T02:21:29.628ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 57 table_name = fields_string
1632026-06-09T02:21:29.629ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 6 table_name = fields_u16
1642026-06-09T02:21:29.630ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 1 table_name = fields_u32
1652026-06-09T02:21:29.631ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 64 table_name = fields_uuid
1662026-06-09T02:21:29.635ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 62 table_name = measurements_cumulativeu64
1672026-06-09T02:21:29.636ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 1 table_name = measurements_f64
1682026-06-09T02:21:29.640ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = measurements_histogramu64
1692026-06-09T02:21:29.641ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 6 table_name = measurements_i64
1702026-06-09T02:21:29.642ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 18 table_name = measurements_u64
1712026-06-09T02:21:29.722ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
1722026-06-09T02:21:29.814ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1732026-06-09T02:21:29.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1742026-06-09T02:21:29.817ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1752026-06-09T02:21:30.815ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1762026-06-09T02:21:30.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1772026-06-09T02:21:30.817ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1782026-06-09T02:21:31.815ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1792026-06-09T02:21:31.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1802026-06-09T02:21:31.817ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1812026-06-09T02:21:31.900ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
1822026-06-09T02:21:31.900ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
1832026-06-09T02:21:31.914ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
1842026-06-09T02:21:32.814ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1852026-06-09T02:21:32.814ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1862026-06-09T02:21:32.817ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1872026-06-09T02:21:33.815ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1882026-06-09T02:21:33.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1892026-06-09T02:21:33.818ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1902026-06-09T02:21:34.617ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 270
1912026-06-09T02:21:34.617ZDEBGoximeter (clickhouse-client): unrolling 270 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
1922026-06-09T02:21:34.622ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 270
1932026-06-09T02:21:34.626ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
1942026-06-09T02:21:34.644ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 280 table_name = fields_i16
1952026-06-09T02:21:34.645ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 771 table_name = fields_string
1962026-06-09T02:21:34.647ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 140 table_name = fields_u16
1972026-06-09T02:21:34.648ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 145 table_name = fields_u32
1982026-06-09T02:21:34.649ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 144 table_name = fields_u8
1992026-06-09T02:21:34.650ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 472 table_name = fields_uuid
2002026-06-09T02:21:34.652ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 4 table_name = measurements_bool
2012026-06-09T02:21:34.653ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 205 table_name = measurements_cumulativeu64
2022026-06-09T02:21:34.654ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 1 table_name = measurements_f64
2032026-06-09T02:21:34.655ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 60 table_name = measurements_u64
2042026-06-09T02:21:34.723ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
2052026-06-09T02:21:34.814ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2062026-06-09T02:21:34.814ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2072026-06-09T02:21:34.817ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2082026-06-09T02:21:35.815ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2092026-06-09T02:21:35.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2102026-06-09T02:21:35.818ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2112026-06-09T02:21:36.815ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2122026-06-09T02:21:36.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2132026-06-09T02:21:36.815ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
2142026-06-09T02:21:36.815ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
2152026-06-09T02:21:36.816ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 2 producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
2162026-06-09T02:21:36.817ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2172026-06-09T02:21:38.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2182026-06-09T02:21:38.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2192026-06-09T02:21:38.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2202026-06-09T02:21:39.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2212026-06-09T02:21:39.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2222026-06-09T02:21:39.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2232026-06-09T02:21:39.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 132
2242026-06-09T02:21:39.823ZDEBGoximeter (clickhouse-client): unrolling 132 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
2252026-06-09T02:21:39.827ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
2262026-06-09T02:21:39.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 132
2272026-06-09T02:21:39.833ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 54 table_name = fields_string
2282026-06-09T02:21:39.834ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = fields_u16
2292026-06-09T02:21:39.835ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 54 table_name = fields_uuid
2302026-06-09T02:21:39.837ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 100 table_name = measurements_cumulativeu64
2312026-06-09T02:21:39.838ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = measurements_histogramu64
2322026-06-09T02:21:39.840ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 30 table_name = measurements_u64
2332026-06-09T02:21:39.929ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
2342026-06-09T02:21:40.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2352026-06-09T02:21:40.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2362026-06-09T02:21:40.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2372026-06-09T02:21:41.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2382026-06-09T02:21:41.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2392026-06-09T02:21:41.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2402026-06-09T02:21:42.009ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
2412026-06-09T02:21:42.009ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending
2422026-06-09T02:21:42.012ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "105f0e52-e452-464d-be44-57d4721a9540", "content-length": "710", "date": "Tue, 09 Jun 2026 02:21:42 GMT"} })
2432026-06-09T02:21:42.012ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D
2442026-06-09T02:21:42.020ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "fef32bc8-1f62-47f9-bea3-b05dba199122", "content-length": "29", "date": "Tue, 09 Jun 2026 02:21:42 GMT"} })
2452026-06-09T02:21:42.020ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:4677 collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601
2462026-06-09T02:21:42.020ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 4 n_pruned_tasks = 0
2472026-06-09T02:21:42.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2482026-06-09T02:21:42.021ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2492026-06-09T02:21:42.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2502026-06-09T02:21:42.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2512026-06-09T02:21:42.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2522026-06-09T02:21:42.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2532026-06-09T02:21:42.031ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2542026-06-09T02:21:42.106ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
2552026-06-09T02:21:42.106ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
2562026-06-09T02:21:42.119ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
2572026-06-09T02:21:43.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2582026-06-09T02:21:43.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2592026-06-09T02:21:43.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2602026-06-09T02:21:43.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2612026-06-09T02:21:43.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2622026-06-09T02:21:43.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2632026-06-09T02:21:44.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2642026-06-09T02:21:44.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2652026-06-09T02:21:44.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2662026-06-09T02:21:44.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2672026-06-09T02:21:44.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2682026-06-09T02:21:44.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2692026-06-09T02:21:44.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 283
2702026-06-09T02:21:44.823ZDEBGoximeter (clickhouse-client): unrolling 283 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
2712026-06-09T02:21:44.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 283
2722026-06-09T02:21:44.832ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
2732026-06-09T02:21:44.836ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("mg_lower:routes_blocked_by_link_state"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:44.826546401Z } timeseries_name = mg_lower:routes_blocked_by_link_state
2742026-06-09T02:21:44.836ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("static_routing_config:static_nexthops"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:44.826425078Z } timeseries_name = static_routing_config:static_nexthops
2752026-06-09T02:21:44.836ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("static_routing_config:static_routes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:44.826365517Z } timeseries_name = static_routing_config:static_routes
2762026-06-09T02:21:44.836ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_rib:active_routes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:44.826485940Z } timeseries_name = switch_rib:active_routes
2772026-06-09T02:21:44.840ZDEBGoximeter (clickhouse-client): inserting 4 new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
2782026-06-09T02:21:44.844ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 280 table_name = fields_i16
2792026-06-09T02:21:44.845ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = fields_ipaddr
2802026-06-09T02:21:44.846ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 776 table_name = fields_string
2812026-06-09T02:21:44.848ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 142 table_name = fields_u16
2822026-06-09T02:21:44.849ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 145 table_name = fields_u32
2832026-06-09T02:21:44.850ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 144 table_name = fields_u8
2842026-06-09T02:21:44.852ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 482 table_name = fields_uuid
2852026-06-09T02:21:44.853ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 4 table_name = measurements_bool
2862026-06-09T02:21:44.854ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 212 table_name = measurements_cumulativeu64
2872026-06-09T02:21:44.855ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 1 table_name = measurements_f64
2882026-06-09T02:21:44.857ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 66 table_name = measurements_u64
2892026-06-09T02:21:44.929ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
2902026-06-09T02:21:45.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2912026-06-09T02:21:45.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2922026-06-09T02:21:45.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2932026-06-09T02:21:45.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2942026-06-09T02:21:45.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2952026-06-09T02:21:45.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2962026-06-09T02:21:46.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2972026-06-09T02:21:46.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2982026-06-09T02:21:46.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2992026-06-09T02:21:46.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3002026-06-09T02:21:46.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3012026-06-09T02:21:46.026ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3022026-06-09T02:21:47.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3032026-06-09T02:21:47.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3042026-06-09T02:21:47.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
3052026-06-09T02:21:47.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3062026-06-09T02:21:47.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3072026-06-09T02:21:47.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
3082026-06-09T02:21:47.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 2 producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
3092026-06-09T02:21:47.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3102026-06-09T02:21:47.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3112026-06-09T02:21:48.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3122026-06-09T02:21:48.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3132026-06-09T02:21:48.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3142026-06-09T02:21:48.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3152026-06-09T02:21:48.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3162026-06-09T02:21:48.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3172026-06-09T02:21:49.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3182026-06-09T02:21:49.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3192026-06-09T02:21:49.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3202026-06-09T02:21:49.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3212026-06-09T02:21:49.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3222026-06-09T02:21:49.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3232026-06-09T02:21:49.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 152
3242026-06-09T02:21:49.823ZDEBGoximeter (clickhouse-client): unrolling 152 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
3252026-06-09T02:21:49.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 152
3262026-06-09T02:21:49.828ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
3272026-06-09T02:21:49.834ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 58 table_name = fields_string
3282026-06-09T02:21:49.835ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = fields_u16
3292026-06-09T02:21:49.837ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 62 table_name = fields_uuid
3302026-06-09T02:21:49.838ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 110 table_name = measurements_cumulativeu64
3312026-06-09T02:21:49.840ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = measurements_histogramu64
3322026-06-09T02:21:49.841ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 40 table_name = measurements_u64
3332026-06-09T02:21:49.929ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
3342026-06-09T02:21:50.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3352026-06-09T02:21:50.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3362026-06-09T02:21:50.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3372026-06-09T02:21:50.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3382026-06-09T02:21:50.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3392026-06-09T02:21:50.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3402026-06-09T02:21:51.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3412026-06-09T02:21:51.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3422026-06-09T02:21:51.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3432026-06-09T02:21:51.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3442026-06-09T02:21:51.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3452026-06-09T02:21:51.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3462026-06-09T02:21:52.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3472026-06-09T02:21:52.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3482026-06-09T02:21:52.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3492026-06-09T02:21:52.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3502026-06-09T02:21:52.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3512026-06-09T02:21:52.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3522026-06-09T02:21:52.107ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
3532026-06-09T02:21:52.107ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
3542026-06-09T02:21:52.118ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 140 producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
3552026-06-09T02:21:53.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3562026-06-09T02:21:53.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3572026-06-09T02:21:53.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3582026-06-09T02:21:53.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3592026-06-09T02:21:53.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3602026-06-09T02:21:53.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3612026-06-09T02:21:54.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3622026-06-09T02:21:54.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3632026-06-09T02:21:54.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3642026-06-09T02:21:54.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3652026-06-09T02:21:54.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3662026-06-09T02:21:54.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3672026-06-09T02:21:54.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 290
3682026-06-09T02:21:54.823ZDEBGoximeter (clickhouse-client): unrolling 290 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
3692026-06-09T02:21:54.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 290
3702026-06-09T02:21:54.832ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
3712026-06-09T02:21:54.842ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 280 table_name = fields_i16
3722026-06-09T02:21:54.844ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 775 table_name = fields_string
3732026-06-09T02:21:54.846ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 140 table_name = fields_u16
3742026-06-09T02:21:54.847ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 145 table_name = fields_u32
3752026-06-09T02:21:54.848ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 144 table_name = fields_u8
3762026-06-09T02:21:54.850ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 480 table_name = fields_uuid
3772026-06-09T02:21:54.851ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 4 table_name = measurements_bool
3782026-06-09T02:21:54.853ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 215 table_name = measurements_cumulativeu64
3792026-06-09T02:21:54.854ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 1 table_name = measurements_f64
3802026-06-09T02:21:54.855ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 70 table_name = measurements_u64
3812026-06-09T02:21:54.929ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
3822026-06-09T02:21:55.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3832026-06-09T02:21:55.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3842026-06-09T02:21:55.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3852026-06-09T02:21:55.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3862026-06-09T02:21:55.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3872026-06-09T02:21:55.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3882026-06-09T02:21:56.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3892026-06-09T02:21:56.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3902026-06-09T02:21:56.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3912026-06-09T02:21:56.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3922026-06-09T02:21:56.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3932026-06-09T02:21:56.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3942026-06-09T02:21:57.008ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
3952026-06-09T02:21:57.009ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending
3962026-06-09T02:21:57.013ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "36273a76-8b06-4e64-91c3-bbbbc9c4452e", "content-length": "710", "date": "Tue, 09 Jun 2026 02:21:57 GMT"} })
3972026-06-09T02:21:57.013ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D
3982026-06-09T02:21:57.016ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "fcd96c73-8002-46d2-8806-c4e27bed6739", "content-length": "29", "date": "Tue, 09 Jun 2026 02:21:57 GMT"} })
3992026-06-09T02:21:57.016ZDEBGoximeter (oximeter-agent): received request to register existing metric producer, updating collection information address = [fd00:1122:3344:101::2]:53700 collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d interval = 30s producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04
4002026-06-09T02:21:57.016ZDEBGoximeter (collection-task-handle): notified collection task of new producer endpoint collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
4012026-06-09T02:21:57.016ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 4 n_pruned_tasks = 0
4022026-06-09T02:21:57.016ZDEBGoximeter (collection-task): collection task received request to update its producer information address = [fd00:1122:3344:101::2]:53700 collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d interval = 30s producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
4032026-06-09T02:21:57.017ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
4042026-06-09T02:21:57.017ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
4052026-06-09T02:21:57.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4062026-06-09T02:21:57.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
4072026-06-09T02:21:57.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4082026-06-09T02:21:57.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
4092026-06-09T02:21:57.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4102026-06-09T02:21:57.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4112026-06-09T02:21:57.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 2 producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
4122026-06-09T02:21:57.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4132026-06-09T02:21:57.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4142026-06-09T02:21:57.069ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 384 producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
4152026-06-09T02:21:58.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4162026-06-09T02:21:58.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4172026-06-09T02:21:58.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4182026-06-09T02:21:58.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4192026-06-09T02:21:58.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4202026-06-09T02:21:58.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4212026-06-09T02:21:59.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4222026-06-09T02:21:59.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4232026-06-09T02:21:59.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4242026-06-09T02:21:59.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4252026-06-09T02:21:59.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4262026-06-09T02:21:59.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4272026-06-09T02:21:59.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 536
4282026-06-09T02:21:59.823ZDEBGoximeter (clickhouse-client): unrolling 536 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
4292026-06-09T02:21:59.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 536
4302026-06-09T02:21:59.840ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
4312026-06-09T02:21:59.844ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:bytes_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:59.827530787Z } timeseries_name = switch_port_control_data_link:bytes_received
4322026-06-09T02:21:59.844ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:bytes_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:59.827655150Z } timeseries_name = switch_port_control_data_link:bytes_sent
4332026-06-09T02:21:59.844ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:errors_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:59.827376403Z } timeseries_name = switch_port_control_data_link:errors_received
4342026-06-09T02:21:59.844ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:errors_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:59.827454895Z } timeseries_name = switch_port_control_data_link:errors_sent
4352026-06-09T02:21:59.844ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:packets_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:59.827593059Z } timeseries_name = switch_port_control_data_link:packets_received
4362026-06-09T02:21:59.844ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:packets_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-06-09T02:21:59.827715121Z } timeseries_name = switch_port_control_data_link:packets_sent
4372026-06-09T02:21:59.848ZDEBGoximeter (clickhouse-client): inserting 6 new timeseries schema collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
4382026-06-09T02:21:59.851ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 24 table_name = fields_i16
4392026-06-09T02:21:59.852ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 178 table_name = fields_string
4402026-06-09T02:21:59.854ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 14 table_name = fields_u16
4412026-06-09T02:21:59.855ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 24 table_name = fields_u32
4422026-06-09T02:21:59.856ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 12 table_name = fields_u8
4432026-06-09T02:21:59.857ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 98 table_name = fields_uuid
4442026-06-09T02:21:59.859ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 494 table_name = measurements_cumulativeu64
4452026-06-09T02:21:59.860ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = measurements_histogramu64
4462026-06-09T02:21:59.862ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 40 table_name = measurements_u64
4472026-06-09T02:21:59.929ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
4482026-06-09T02:22:00.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4492026-06-09T02:22:00.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4502026-06-09T02:22:00.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4512026-06-09T02:22:00.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4522026-06-09T02:22:00.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4532026-06-09T02:22:00.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4542026-06-09T02:22:01.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4552026-06-09T02:22:01.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4562026-06-09T02:22:01.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4572026-06-09T02:22:01.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4582026-06-09T02:22:01.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4592026-06-09T02:22:01.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4602026-06-09T02:22:02.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4612026-06-09T02:22:02.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4622026-06-09T02:22:02.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4632026-06-09T02:22:02.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4642026-06-09T02:22:02.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4652026-06-09T02:22:02.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4662026-06-09T02:22:03.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4672026-06-09T02:22:03.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4682026-06-09T02:22:03.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4692026-06-09T02:22:03.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4702026-06-09T02:22:03.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4712026-06-09T02:22:03.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4722026-06-09T02:22:04.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4732026-06-09T02:22:04.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4742026-06-09T02:22:04.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4752026-06-09T02:22:04.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4762026-06-09T02:22:04.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4772026-06-09T02:22:04.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4782026-06-09T02:22:04.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 150
4792026-06-09T02:22:04.823ZDEBGoximeter (clickhouse-client): unrolling 150 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
4802026-06-09T02:22:04.828ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
4812026-06-09T02:22:04.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 150
4822026-06-09T02:22:04.833ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 54 table_name = fields_string
4832026-06-09T02:22:04.835ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 60 table_name = fields_uuid
4842026-06-09T02:22:04.836ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 110 table_name = measurements_cumulativeu64
4852026-06-09T02:22:04.837ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 40 table_name = measurements_u64
4862026-06-09T02:22:04.928ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
4872026-06-09T02:22:05.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4882026-06-09T02:22:05.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4892026-06-09T02:22:05.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4902026-06-09T02:22:05.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4912026-06-09T02:22:05.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4922026-06-09T02:22:05.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4932026-06-09T02:22:06.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4942026-06-09T02:22:06.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4952026-06-09T02:22:06.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4962026-06-09T02:22:06.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4972026-06-09T02:22:06.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4982026-06-09T02:22:06.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4992026-06-09T02:22:07.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5002026-06-09T02:22:07.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5012026-06-09T02:22:07.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
5022026-06-09T02:22:07.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
5032026-06-09T02:22:07.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5042026-06-09T02:22:07.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5052026-06-09T02:22:07.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5062026-06-09T02:22:07.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 2 producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
5072026-06-09T02:22:07.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5082026-06-09T02:22:08.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5092026-06-09T02:22:08.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5102026-06-09T02:22:08.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5112026-06-09T02:22:08.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5122026-06-09T02:22:08.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5132026-06-09T02:22:08.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5142026-06-09T02:22:09.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5152026-06-09T02:22:09.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5162026-06-09T02:22:09.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5172026-06-09T02:22:09.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5182026-06-09T02:22:09.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5192026-06-09T02:22:09.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5202026-06-09T02:22:09.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 152
5212026-06-09T02:22:09.823ZDEBGoximeter (clickhouse-client): unrolling 152 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
5222026-06-09T02:22:09.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 152
5232026-06-09T02:22:09.828ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
5242026-06-09T02:22:09.834ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 58 table_name = fields_string
5252026-06-09T02:22:09.835ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = fields_u16
5262026-06-09T02:22:09.837ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 62 table_name = fields_uuid
5272026-06-09T02:22:09.838ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 110 table_name = measurements_cumulativeu64
5282026-06-09T02:22:09.840ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = measurements_histogramu64
5292026-06-09T02:22:09.841ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 40 table_name = measurements_u64
5302026-06-09T02:22:09.928ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
5312026-06-09T02:22:10.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5322026-06-09T02:22:10.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5332026-06-09T02:22:10.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5342026-06-09T02:22:10.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5352026-06-09T02:22:10.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5362026-06-09T02:22:10.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5372026-06-09T02:22:11.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5382026-06-09T02:22:11.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5392026-06-09T02:22:11.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5402026-06-09T02:22:11.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5412026-06-09T02:22:11.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5422026-06-09T02:22:11.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5432026-06-09T02:22:12.009ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
5442026-06-09T02:22:12.009ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending
5452026-06-09T02:22:12.014ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "670d0559-7761-40d7-a287-eaa911e52a26", "content-length": "710", "date": "Tue, 09 Jun 2026 02:22:12 GMT"} })
5462026-06-09T02:22:12.014ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D
5472026-06-09T02:22:12.017ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "7c4d9e51-deee-4a24-83ed-a465e6ec82ee", "content-length": "29", "date": "Tue, 09 Jun 2026 02:22:12 GMT"} })
5482026-06-09T02:22:12.017ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 4 n_pruned_tasks = 0
5492026-06-09T02:22:12.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5502026-06-09T02:22:12.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5512026-06-09T02:22:12.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5522026-06-09T02:22:12.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5532026-06-09T02:22:12.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5542026-06-09T02:22:12.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5552026-06-09T02:22:12.096ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
5562026-06-09T02:22:13.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5572026-06-09T02:22:13.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5582026-06-09T02:22:13.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5592026-06-09T02:22:13.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5602026-06-09T02:22:13.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5612026-06-09T02:22:13.025ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5622026-06-09T02:22:14.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5632026-06-09T02:22:14.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5642026-06-09T02:22:14.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5652026-06-09T02:22:14.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5662026-06-09T02:22:14.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5672026-06-09T02:22:14.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5682026-06-09T02:22:14.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 151
5692026-06-09T02:22:14.823ZDEBGoximeter (clickhouse-client): unrolling 151 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
5702026-06-09T02:22:14.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 151
5712026-06-09T02:22:14.828ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
5722026-06-09T02:22:14.834ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = fields_ipaddr
5732026-06-09T02:22:14.835ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 55 table_name = fields_string
5742026-06-09T02:22:14.837ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 2 table_name = fields_u16
5752026-06-09T02:22:14.838ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 62 table_name = fields_uuid
5762026-06-09T02:22:14.839ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 111 table_name = measurements_cumulativeu64
5772026-06-09T02:22:14.840ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 40 table_name = measurements_u64
5782026-06-09T02:22:14.929ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
5792026-06-09T02:22:15.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5802026-06-09T02:22:15.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5812026-06-09T02:22:15.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5822026-06-09T02:22:15.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5832026-06-09T02:22:15.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5842026-06-09T02:22:15.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5852026-06-09T02:22:16.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5862026-06-09T02:22:16.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5872026-06-09T02:22:16.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5882026-06-09T02:22:16.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5892026-06-09T02:22:16.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5902026-06-09T02:22:16.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5912026-06-09T02:22:17.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5922026-06-09T02:22:17.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5932026-06-09T02:22:17.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
5942026-06-09T02:22:17.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
5952026-06-09T02:22:17.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5962026-06-09T02:22:17.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
5972026-06-09T02:22:17.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
5982026-06-09T02:22:17.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 3 producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
5992026-06-09T02:22:17.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6002026-06-09T02:22:18.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6012026-06-09T02:22:18.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6022026-06-09T02:22:18.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6032026-06-09T02:22:18.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6042026-06-09T02:22:18.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6052026-06-09T02:22:18.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6062026-06-09T02:22:19.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6072026-06-09T02:22:19.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6082026-06-09T02:22:19.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6092026-06-09T02:22:19.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6102026-06-09T02:22:19.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6112026-06-09T02:22:19.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6122026-06-09T02:22:19.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 153
6132026-06-09T02:22:19.823ZDEBGoximeter (clickhouse-client): unrolling 153 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
6142026-06-09T02:22:19.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 153
6152026-06-09T02:22:19.828ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
6162026-06-09T02:22:19.834ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 60 table_name = fields_string
6172026-06-09T02:22:19.835ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 3 table_name = fields_u16
6182026-06-09T02:22:19.836ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 63 table_name = fields_uuid
6192026-06-09T02:22:19.838ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 110 table_name = measurements_cumulativeu64
6202026-06-09T02:22:19.839ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 3 table_name = measurements_histogramu64
6212026-06-09T02:22:19.841ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 40 table_name = measurements_u64
6222026-06-09T02:22:19.929ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
6232026-06-09T02:22:20.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6242026-06-09T02:22:20.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6252026-06-09T02:22:20.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6262026-06-09T02:22:20.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6272026-06-09T02:22:20.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6282026-06-09T02:22:20.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6292026-06-09T02:22:21.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6302026-06-09T02:22:21.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6312026-06-09T02:22:21.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6322026-06-09T02:22:21.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6332026-06-09T02:22:21.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6342026-06-09T02:22:21.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6352026-06-09T02:22:22.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6362026-06-09T02:22:22.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6372026-06-09T02:22:22.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6382026-06-09T02:22:22.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6392026-06-09T02:22:22.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6402026-06-09T02:22:22.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6412026-06-09T02:22:23.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6422026-06-09T02:22:23.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6432026-06-09T02:22:23.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6442026-06-09T02:22:23.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6452026-06-09T02:22:23.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6462026-06-09T02:22:23.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6472026-06-09T02:22:24.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6482026-06-09T02:22:24.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6492026-06-09T02:22:24.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6502026-06-09T02:22:24.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6512026-06-09T02:22:24.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6522026-06-09T02:22:24.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6532026-06-09T02:22:24.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 150
6542026-06-09T02:22:24.823ZDEBGoximeter (clickhouse-client): unrolling 150 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
6552026-06-09T02:22:24.827ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 150
6562026-06-09T02:22:24.829ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
6572026-06-09T02:22:24.835ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 54 table_name = fields_string
6582026-06-09T02:22:24.836ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 60 table_name = fields_uuid
6592026-06-09T02:22:24.838ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 110 table_name = measurements_cumulativeu64
6602026-06-09T02:22:24.839ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 40 table_name = measurements_u64
6612026-06-09T02:22:24.929ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
6622026-06-09T02:22:25.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6632026-06-09T02:22:25.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6642026-06-09T02:22:25.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6652026-06-09T02:22:25.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6662026-06-09T02:22:25.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6672026-06-09T02:22:25.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6682026-06-09T02:22:26.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6692026-06-09T02:22:26.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6702026-06-09T02:22:26.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6712026-06-09T02:22:26.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6722026-06-09T02:22:26.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6732026-06-09T02:22:26.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6742026-06-09T02:22:27.009ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:565
6752026-06-09T02:22:27.009ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending
6762026-06-09T02:22:27.014ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "789658e0-867a-4d64-b6e8-0dbe18725b02", "content-length": "1122", "date": "Tue, 09 Jun 2026 02:22:27 GMT"} })
6772026-06-09T02:22:27.014ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D
6782026-06-09T02:22:27.017ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
6792026-06-09T02:22:27.017ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
6802026-06-09T02:22:27.017ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/e341817b-fe8a-48ae-9a66-66ec8c47e844/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJlYTdkMDZmYS0xMTRlLTQ4NWUtYWJlYy05ZjczOGYwYTY4ZjcifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "78aead76-d854-4e72-af28-3cb83fed54b5", "content-length": "29", "date": "Tue, 09 Jun 2026 02:22:27 GMT"} })
6812026-06-09T02:22:27.017ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::a]:51254 collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 5a7052cd-ea15-4706-ad66-604c31f239a9
6822026-06-09T02:22:27.017ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::b]:48721 collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 79b3cedd-cd80-4b2c-9704-89b7545bbb9d
6832026-06-09T02:22:27.018ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:8001 collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77
6842026-06-09T02:22:27.018ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:640 n_current_tasks = 7 n_pruned_tasks = 0
6852026-06-09T02:22:27.019ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
6862026-06-09T02:22:27.019ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 5a7052cd-ea15-4706-ad66-604c31f239a9 producer_ip = fd00:1122:3344:101::a producer_port = 51254
6872026-06-09T02:22:27.019ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 5a7052cd-ea15-4706-ad66-604c31f239a9 producer_ip = fd00:1122:3344:101::a producer_port = 51254
6882026-06-09T02:22:27.019ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 5a7052cd-ea15-4706-ad66-604c31f239a9 producer_ip = fd00:1122:3344:101::a producer_port = 51254
6892026-06-09T02:22:27.019ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 79b3cedd-cd80-4b2c-9704-89b7545bbb9d producer_ip = fd00:1122:3344:101::b producer_port = 48721
6902026-06-09T02:22:27.019ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 79b3cedd-cd80-4b2c-9704-89b7545bbb9d producer_ip = fd00:1122:3344:101::b producer_port = 48721
6912026-06-09T02:22:27.019ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 79b3cedd-cd80-4b2c-9704-89b7545bbb9d producer_ip = fd00:1122:3344:101::b producer_port = 48721
6922026-06-09T02:22:27.019ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
6932026-06-09T02:22:27.019ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
6942026-06-09T02:22:27.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 36 producer_id = 632beda4-8f21-427f-a422-eee1da8b6b04 producer_ip = fd00:1122:3344:101::2 producer_port = 52706
6952026-06-09T02:22:27.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6962026-06-09T02:22:27.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
6972026-06-09T02:22:27.021ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
6982026-06-09T02:22:27.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
6992026-06-09T02:22:27.021ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7002026-06-09T02:22:27.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7012026-06-09T02:22:27.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
7022026-06-09T02:22:27.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
7032026-06-09T02:22:27.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7042026-06-09T02:22:27.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 3 producer_id = 1bf57f6b-e73a-4338-a384-5d2c11ddb2c1 producer_ip = fd00:1122:3344:101::c producer_port = 43239
7052026-06-09T02:22:27.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7062026-06-09T02:22:27.027ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 10 producer_id = 5a7052cd-ea15-4706-ad66-604c31f239a9 producer_ip = fd00:1122:3344:101::a producer_port = 51254
7072026-06-09T02:22:27.027ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7082026-06-09T02:22:27.028ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 5 n_samples = 5 producer_id = 79b3cedd-cd80-4b2c-9704-89b7545bbb9d producer_ip = fd00:1122:3344:101::b producer_port = 48721
7092026-06-09T02:22:28.019ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7102026-06-09T02:22:28.019ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7112026-06-09T02:22:28.020ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7122026-06-09T02:22:28.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7132026-06-09T02:22:28.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7142026-06-09T02:22:28.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7152026-06-09T02:22:28.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7162026-06-09T02:22:28.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7172026-06-09T02:22:28.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7182026-06-09T02:22:29.019ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7192026-06-09T02:22:29.019ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7202026-06-09T02:22:29.020ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7212026-06-09T02:22:29.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7222026-06-09T02:22:29.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7232026-06-09T02:22:29.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7242026-06-09T02:22:29.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7252026-06-09T02:22:29.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7262026-06-09T02:22:29.024ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7272026-06-09T02:22:29.823ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 251
7282026-06-09T02:22:29.823ZDEBGoximeter (clickhouse-client): unrolling 251 total samples collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
7292026-06-09T02:22:29.828ZDEBGoximeter (database-inserter): inserting samples into database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_samples = 251
7302026-06-09T02:22:29.831ZDEBGoximeter (clickhouse-client): fetching timeseries schema from database collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07
7312026-06-09T02:22:29.839ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 24 table_name = fields_i16
7322026-06-09T02:22:29.840ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 10 table_name = fields_ipaddr
7332026-06-09T02:22:29.842ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 228 table_name = fields_string
7342026-06-09T02:22:29.844ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 33 table_name = fields_u16
7352026-06-09T02:22:29.845ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 25 table_name = fields_u32
7362026-06-09T02:22:29.846ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 12 table_name = fields_u8
7372026-06-09T02:22:29.847ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 151 table_name = fields_uuid
7382026-06-09T02:22:29.848ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 181 table_name = measurements_cumulativeu64
7392026-06-09T02:22:29.850ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 1 table_name = measurements_f64
7402026-06-09T02:22:29.852ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 11 table_name = measurements_histogramu64
7412026-06-09T02:22:29.853ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 6 table_name = measurements_i64
7422026-06-09T02:22:29.854ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d id = 279ada81-1da3-4fb0-87f1-aacb99b3ed07 n_rows = 52 table_name = measurements_u64
7432026-06-09T02:22:29.929ZWARNoximeter (database-inserter): failed to insert some results into metric DB collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:269
7442026-06-09T02:22:30.018ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7452026-06-09T02:22:30.018ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7462026-06-09T02:22:30.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7472026-06-09T02:22:30.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7482026-06-09T02:22:30.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7492026-06-09T02:22:30.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7502026-06-09T02:22:30.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7512026-06-09T02:22:30.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7522026-06-09T02:22:30.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7532026-06-09T02:22:31.019ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7542026-06-09T02:22:31.019ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7552026-06-09T02:22:31.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7562026-06-09T02:22:31.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7572026-06-09T02:22:31.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7582026-06-09T02:22:31.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7592026-06-09T02:22:31.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7602026-06-09T02:22:31.022ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7612026-06-09T02:22:31.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7622026-06-09T02:22:32.019ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7632026-06-09T02:22:32.019ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7642026-06-09T02:22:32.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7652026-06-09T02:22:32.021ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7662026-06-09T02:22:32.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7672026-06-09T02:22:32.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7682026-06-09T02:22:32.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7692026-06-09T02:22:32.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7702026-06-09T02:22:32.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7712026-06-09T02:22:33.019ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7722026-06-09T02:22:33.019ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7732026-06-09T02:22:33.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7742026-06-09T02:22:33.020ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7752026-06-09T02:22:33.020ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7762026-06-09T02:22:33.021ZDEBGoximeter (collection-task): collecting from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
7772026-06-09T02:22:33.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 4 producer_id = 62b3ae1d-9344-4c91-ae03-0c204eb99601 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
7782026-06-09T02:22:33.021ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 14 producer_id = 7e6ce80b-337f-496d-b3ed-2fe1ce2b3a77 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
7792026-06-09T02:22:33.023ZDEBGoximeter (collection-task): collected results from producer collector_id = e341817b-fe8a-48ae-9a66-66ec8c47e844 collector_ip = fd00:1122:3344:101::d n_results = 1 n_samples = 26 producer_id = ea7d06fa-114e-485e-abec-9f738f0a68f7 producer_ip = fd00:1122:3344:101::1 producer_port = 8001