Max level shown:
1 [ Mar 3 19:14:12 Enabled. ]
2 [ Mar 3 19:14:12 Rereading configuration. ]
3 [ Mar 3 19:14:14 Rereading configuration. ]
4 [ Mar 3 19:14:22 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/oximeter-collector/bin/oximeter run /var/svc/manifest/site/oximeter/config.toml --address [fd00:1122:3344:101::d]:12223 --id d37cbb5f-1c84-406e-84dd-3efb89358613 &"). ]
5 [ Mar 3 19:14:22 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72026-03-03T19:14:23.030ZDEBGoximeter: registered DTrace probes
82026-03-03T19:14:23.070ZINFOoximeter: starting oximeter server file = oximeter/collector/src/lib.rs:264
92026-03-03T19:14:23.081ZDEBGoximeter: creating ClickHouse client
102026-03-03T19:14:23.112ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
112026-03-03T19:14:32.989ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:900 id = ce417871-7805-44c3-bf7e-14ebf076bb74
122026-03-03T19:14:32.989ZDEBGoximeter (oximeter-agent): oximeter database does not exist, creating collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d
132026-03-03T19:14:32.989ZINFOoximeter (clickhouse-client): reading db version collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:800 id = ce417871-7805-44c3-bf7e-14ebf076bb74
142026-03-03T19:14:32.991ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:900 id = ce417871-7805-44c3-bf7e-14ebf076bb74
152026-03-03T19:14:32.991ZINFOoximeter (clickhouse-client): read oximeter database version collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:805 id = ce417871-7805-44c3-bf7e-14ebf076bb74 version = 0
162026-03-03T19:14:32.991ZINFOoximeter (clickhouse-client): initializing oximeter schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:810 id = ce417871-7805-44c3-bf7e-14ebf076bb74
172026-03-03T19:14:32.991ZDEBGoximeter (clickhouse-client): initializing ClickHouse database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
182026-03-03T19:14:33.921ZINFOoximeter (clickhouse-client): inserting current version collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:831 id = ce417871-7805-44c3-bf7e-14ebf076bb74 version = 14
192026-03-03T19:14:33.952ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = 640c9ad8-87e2-4345-8698-468409f9f9ce
202026-03-03T19:14:33.966ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /info versions = all
212026-03-03T19:14:33.966ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers versions = all
222026-03-03T19:14:33.966ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = DELETE path = /producers/{producer_id} versions = all
232026-03-03T19:14:33.966ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers/{producer_id} versions = all
242026-03-03T19:14:33.966ZINFOoximeter (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::d]:12223
252026-03-03T19:14:33.979ZDEBGoximeter (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::d]:12223
262026-03-03T19:14:33.979ZDEBGoximeter: registered USDT probes
272026-03-03T19:14:33.979ZDEBGoximeter: contacting nexus
282026-03-03T19:14:34.092ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors
292026-03-03T19:14:34.098ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::b]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
302026-03-03T19:14:34.098ZWARNoximeter: failed to contact nexus, will retry in 174.649583ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:409
312026-03-03T19:14:34.274ZDEBGoximeter: contacting nexus
322026-03-03T19:14:34.274ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors
332026-03-03T19:14:34.274ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::c]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
342026-03-03T19:14:34.274ZWARNoximeter: failed to contact nexus, will retry in 629.976279ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::c]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:409
352026-03-03T19:14:34.905ZDEBGoximeter: contacting nexus
362026-03-03T19:14:34.905ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors
372026-03-03T19:14:34.906ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::b]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
382026-03-03T19:14:34.906ZWARNoximeter: failed to contact nexus, will retry in 572.74796ms error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:409
392026-03-03T19:14:35.480ZDEBGoximeter: contacting nexus
402026-03-03T19:14:35.480ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors
412026-03-03T19:14:35.480ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::a]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
422026-03-03T19:14:35.480ZWARNoximeter: failed to contact nexus, will retry in 2.857270233s error = "Communication Error: error sending request for url (http://[fd00:1122:3344:101::a]:12221/metrics/collectors)" file = oximeter/collector/src/lib.rs:409
432026-03-03T19:14:38.339ZDEBGoximeter: contacting nexus
442026-03-03T19:14:38.339ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors
452026-03-03T19:14:38.458ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors", status: 204, headers: {"x-request-id": "520b90ec-1501-43ea-ade1-4f41e78a5f5b", "date": "Tue, 03 Mar 2026 19:14:38 GMT"} })
462026-03-03T19:14:38.458ZINFOoximeter: oximeter registered with nexus file = oximeter/collector/src/lib.rs:427 id = d37cbb5f-1c84-406e-84dd-3efb89358613
472026-03-03T19:14:38.458ZINFOoximeter (oximeter-agent): starting refresh list task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:554
482026-03-03T19:14:38.459ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
492026-03-03T19:14:38.466ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::b]:12221/metrics/collectors/d37cbb5f-1c84-406e-84dd-3efb89358613/producers?sort_by=id_ascending
502026-03-03T19:14:38.466ZDEBGoximeter: client response result = Err(reqwest::Error { kind: Request, url: "http://[fd00:1122:3344:101::b]:12221/metrics/collectors/d37cbb5f-1c84-406e-84dd-3efb89358613/producers?sort_by=id_ascending", source: hyper_util::client::legacy::Error(Connect, ConnectError("tcp connect error", [fd00:1122:3344:101::b]:12221, Os { code: 146, kind: ConnectionRefused, message: "Connection refused" })) })
512026-03-03T19:14:38.469ZERROoximeter (oximeter-agent): error fetching next assigned producer, abandoning this refresh attempt collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d error = Communication Error: error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/collectors/d37cbb5f-1c84-406e-84dd-3efb89358613/producers?sort_by=id_ascending): error sending request for url (http://[fd00:1122:3344:101::b]:12221/metrics/collectors/d37cbb5f-1c84-406e-84dd-3efb89358613/producers?sort_by=id_ascending): client error (Connect): tcp connect error: Connection refused (os error 146) file = oximeter/collector/src/agent.rs:595
522026-03-03T19:14:53.462ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
532026-03-03T19:14:53.462ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d37cbb5f-1c84-406e-84dd-3efb89358613/producers?sort_by=id_ascending
542026-03-03T19:14:53.620ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d37cbb5f-1c84-406e-84dd-3efb89358613/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "c372fc13-03b2-4021-8979-ffab03c327c1", "content-length": "572", "date": "Tue, 03 Mar 2026 19:14:53 GMT"} })
552026-03-03T19:14:53.627ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d37cbb5f-1c84-406e-84dd-3efb89358613/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhZDA2ZmEwMi0yM2ZlLTQxODctOWE1Yi1iNTkzMmJkNGU0YzUifX0%3D
562026-03-03T19:14:53.738ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d37cbb5f-1c84-406e-84dd-3efb89358613/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhZDA2ZmEwMi0yM2ZlLTQxODctOWE1Yi1iNTkzMmJkNGU0YzUifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "5bbe0f44-3861-4775-8b9f-a08dad751b36", "content-length": "29", "date": "Tue, 03 Mar 2026 19:14:53 GMT"} })
572026-03-03T19:14:53.744ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:8001 collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69
582026-03-03T19:14:53.746ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:4677 collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca
592026-03-03T19:14:53.746ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:51321 collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = ad06fa02-23fe-4187-9a5b-b5932bd4e4c5
602026-03-03T19:14:53.746ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 3 n_pruned_tasks = 0
612026-03-03T19:14:53.751ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
622026-03-03T19:14:53.751ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = ad06fa02-23fe-4187-9a5b-b5932bd4e4c5 producer_ip = fd00:1122:3344:101::2 producer_port = 51321
632026-03-03T19:14:53.751ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
642026-03-03T19:14:53.789ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
652026-03-03T19:14:53.789ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
662026-03-03T19:14:53.789ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = ad06fa02-23fe-4187-9a5b-b5932bd4e4c5 producer_ip = fd00:1122:3344:101::2 producer_port = 51321
672026-03-03T19:14:53.789ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
682026-03-03T19:14:53.789ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
692026-03-03T19:14:53.789ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = ad06fa02-23fe-4187-9a5b-b5932bd4e4c5 producer_ip = fd00:1122:3344:101::2 producer_port = 51321
702026-03-03T19:14:53.813ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
712026-03-03T19:14:53.814ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
722026-03-03T19:14:53.823ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = ad06fa02-23fe-4187-9a5b-b5932bd4e4c5 producer_ip = fd00:1122:3344:101::2 producer_port = 51321
732026-03-03T19:14:53.957ZDEBGoximeter (results-sink): inserting 175 samples into database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d
742026-03-03T19:14:53.957ZDEBGoximeter (results-sink-cluster): inserting 175 samples into database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d
752026-03-03T19:14:53.957ZDEBGoximeter (clickhouse-client): unrolling 175 total samples collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
762026-03-03T19:14:53.958ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
772026-03-03T19:14:53.999ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:collections"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "base_route", field_type: String, source: Metric, description: "" }, FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "producer_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "producer_ip", field_type: IpAddr, source: Metric, description: "" }, FieldSchema { name: "producer_port", field_type: U16, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:53.958051599Z } timeseries_name = oximeter_collector:collections
782026-03-03T19:14:54.013ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
792026-03-03T19:14:54.037ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("static_routing_config:static_routes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.013664163Z } timeseries_name = static_routing_config:static_routes
802026-03-03T19:14:54.037ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
812026-03-03T19:14:54.043ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("static_routing_config:static_nexthops"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.037930813Z } timeseries_name = static_routing_config:static_nexthops
822026-03-03T19:14:54.043ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
832026-03-03T19:14:54.044ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_rib:active_routes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.043846538Z } timeseries_name = switch_rib:active_routes
842026-03-03T19:14:54.044ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
852026-03-03T19:14:54.045ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("mg_lower:routes_blocked_by_link_state"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.044904695Z } timeseries_name = mg_lower:routes_blocked_by_link_state
862026-03-03T19:14:54.045ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
872026-03-03T19:14:54.046ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.045876240Z } timeseries_name = ddm_router:originated_underlay_prefixes
882026-03-03T19:14:54.046ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
892026-03-03T19:14:54.047ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.046811106Z } timeseries_name = ddm_router:originated_tunnel_endpoints
902026-03-03T19:14:54.047ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
912026-03-03T19:14:54.048ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.047827362Z } timeseries_name = ddm_session:solicitations_sent
922026-03-03T19:14:54.048ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
932026-03-03T19:14:54.049ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.048772157Z } timeseries_name = ddm_session:solicitations_received
942026-03-03T19:14:54.049ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
952026-03-03T19:14:54.050ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.049791043Z } timeseries_name = ddm_session:advertisements_sent
962026-03-03T19:14:54.050ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
972026-03-03T19:14:54.051ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.050820850Z } timeseries_name = ddm_session:advertisements_received
982026-03-03T19:14:54.051ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
992026-03-03T19:14:54.052ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_expirations"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.051881797Z } timeseries_name = ddm_session:peer_expirations
1002026-03-03T19:14:54.052ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1012026-03-03T19:14:54.053ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_address_changes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.052871763Z } timeseries_name = ddm_session:peer_address_changes
1022026-03-03T19:14:54.053ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1032026-03-03T19:14:54.054ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_sessions_established"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.053848948Z } timeseries_name = ddm_session:peer_sessions_established
1042026-03-03T19:14:54.054ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1052026-03-03T19:14:54.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.054995377Z } timeseries_name = ddm_session:updates_sent
1062026-03-03T19:14:54.055ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1072026-03-03T19:14:54.056ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.055976803Z } timeseries_name = ddm_session:updates_received
1082026-03-03T19:14:54.056ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1092026-03-03T19:14:54.057ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
1102026-03-03T19:14:54.058ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:update_send_fail"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.056939458Z } timeseries_name = ddm_session:update_send_fail
1112026-03-03T19:14:54.058ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1122026-03-03T19:14:54.059ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.058169428Z } timeseries_name = ddm_session:imported_underlay_prefixes
1132026-03-03T19:14:54.059ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1142026-03-03T19:14:54.060ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.059227055Z } timeseries_name = ddm_session:imported_tunnel_endpoints
1152026-03-03T19:14:54.060ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1162026-03-03T19:14:54.061ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_data_link:link_enabled"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_id", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "port_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: Bool, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.060671998Z } timeseries_name = switch_data_link:link_enabled
1172026-03-03T19:14:54.064ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1182026-03-03T19:14:54.065ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_data_link:link_up"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_id", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "port_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: Bool, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.064210175Z } timeseries_name = switch_data_link:link_up
1192026-03-03T19:14:54.065ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1202026-03-03T19:14:54.066ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:capacity"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.065535786Z } timeseries_name = switch_table:capacity
1212026-03-03T19:14:54.066ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1222026-03-03T19:14:54.067ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:occupancy"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.066569473Z } timeseries_name = switch_table:occupancy
1232026-03-03T19:14:54.067ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1242026-03-03T19:14:54.068ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:inserts"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.067576269Z } timeseries_name = switch_table:inserts
1252026-03-03T19:14:54.068ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1262026-03-03T19:14:54.069ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:deletes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.068603856Z } timeseries_name = switch_table:deletes
1272026-03-03T19:14:54.069ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1282026-03-03T19:14:54.070ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:updates"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.069608682Z } timeseries_name = switch_table:updates
1292026-03-03T19:14:54.070ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1302026-03-03T19:14:54.071ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:collisions"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.070612748Z } timeseries_name = switch_table:collisions
1312026-03-03T19:14:54.071ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1322026-03-03T19:14:54.072ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:update_misses"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.071713356Z } timeseries_name = switch_table:update_misses
1332026-03-03T19:14:54.072ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1342026-03-03T19:14:54.073ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:delete_misses"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.072813393Z } timeseries_name = switch_table:delete_misses
1352026-03-03T19:14:54.073ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1362026-03-03T19:14:54.074ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:exhaustion"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.073955502Z } timeseries_name = switch_table:exhaustion
1372026-03-03T19:14:54.082ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1382026-03-03T19:14:54.083ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 schema = TimeseriesSchema { timeseries_name: TimeseriesName("dendrite:sample_collection_duration"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Metric, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Metric, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Metric, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Metric, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Metric, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Metric, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Metric, description: "" }}, datum_type: F64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-03T19:14:54.082656462Z } timeseries_name = dendrite:sample_collection_duration
1392026-03-03T19:14:54.083ZDEBGoximeter (clickhouse-client): inserting 31 new timeseries schema collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1402026-03-03T19:14:54.104ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 284 table_name = fields_i16
1412026-03-03T19:14:54.106ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 6 table_name = fields_ipaddr
1422026-03-03T19:14:54.108ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 774 table_name = fields_string
1432026-03-03T19:14:54.110ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 148 table_name = fields_u16
1442026-03-03T19:14:54.112ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 149 table_name = fields_u32
1452026-03-03T19:14:54.116ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 148 table_name = fields_u8
1462026-03-03T19:14:54.119ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 468 table_name = fields_uuid
1472026-03-03T19:14:54.125ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 6 table_name = measurements_bool
1482026-03-03T19:14:54.130ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 130 table_name = measurements_cumulativeu64
1492026-03-03T19:14:54.132ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 1 table_name = measurements_f64
1502026-03-03T19:14:54.134ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 38 table_name = measurements_u64
1512026-03-03T19:14:54.745ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1522026-03-03T19:14:54.745ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1532026-03-03T19:14:54.747ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1542026-03-03T19:14:54.747ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1552026-03-03T19:14:54.748ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1562026-03-03T19:14:54.749ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1572026-03-03T19:14:55.745ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1582026-03-03T19:14:55.745ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1592026-03-03T19:14:55.746ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1602026-03-03T19:14:55.746ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1612026-03-03T19:14:55.747ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1622026-03-03T19:14:55.748ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1632026-03-03T19:14:56.745ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1642026-03-03T19:14:56.745ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1652026-03-03T19:14:56.746ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1662026-03-03T19:14:56.746ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1672026-03-03T19:14:56.747ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1682026-03-03T19:14:56.748ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1692026-03-03T19:14:57.745ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1702026-03-03T19:14:57.745ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1712026-03-03T19:14:57.747ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1722026-03-03T19:14:57.747ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1732026-03-03T19:14:57.747ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1742026-03-03T19:14:57.749ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1752026-03-03T19:14:58.745ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1762026-03-03T19:14:58.745ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1772026-03-03T19:14:58.747ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1782026-03-03T19:14:58.747ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1792026-03-03T19:14:58.747ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1802026-03-03T19:14:58.748ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1812026-03-03T19:14:58.956ZDEBGoximeter (results-sink-cluster): inserting 150 samples into database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d
1822026-03-03T19:14:58.956ZDEBGoximeter (results-sink): inserting 150 samples into database collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d
1832026-03-03T19:14:58.957ZDEBGoximeter (clickhouse-client): unrolling 150 total samples collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74
1842026-03-03T19:14:58.964ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 30 table_name = fields_string
1852026-03-03T19:14:58.966ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 36 table_name = fields_uuid
1862026-03-03T19:14:58.967ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 110 table_name = measurements_cumulativeu64
1872026-03-03T19:14:58.969ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d id = ce417871-7805-44c3-bf7e-14ebf076bb74 n_rows = 40 table_name = measurements_u64
1882026-03-03T19:14:59.058ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
1892026-03-03T19:14:59.745ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1902026-03-03T19:14:59.745ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1912026-03-03T19:14:59.746ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1922026-03-03T19:14:59.746ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1932026-03-03T19:14:59.747ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1942026-03-03T19:14:59.748ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1952026-03-03T19:15:00.745ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1962026-03-03T19:15:00.745ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1972026-03-03T19:15:00.747ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1982026-03-03T19:15:00.747ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1992026-03-03T19:15:00.747ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2002026-03-03T19:15:00.748ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2012026-03-03T19:15:01.745ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2022026-03-03T19:15:01.745ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2032026-03-03T19:15:01.747ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2042026-03-03T19:15:01.747ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2052026-03-03T19:15:01.748ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2062026-03-03T19:15:01.748ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2072026-03-03T19:15:02.745ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2082026-03-03T19:15:02.745ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2092026-03-03T19:15:02.746ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2102026-03-03T19:15:02.747ZDEBGoximeter (collection-task): collecting from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2112026-03-03T19:15:02.747ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 75abf3a4-8fd5-4d22-ad7e-db197f03b3ca producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2122026-03-03T19:15:02.748ZDEBGoximeter (collection-task): collected results from producer collector_id = d37cbb5f-1c84-406e-84dd-3efb89358613 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 16531efa-0062-408b-ab82-d5742995ed69 producer_ip = fd00:1122:3344:101::1 producer_port = 8001