Max level shown:
1 [ Mar 24 22:28:49 Enabled. ]
2 [ Mar 24 22:28:49 Rereading configuration. ]
3 [ Mar 24 22:28:51 Rereading configuration. ]
4 [ Mar 24 22:28:56 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/oximeter-collector/bin/oximeter run /var/svc/manifest/site/oximeter/config.toml --address [fd00:1122:3344:101::d]:12223 --id d008b716-c031-4c28-af42-a025bc80f52e &"). ]
5 [ Mar 24 22:28:56 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72026-03-24T22:28:56.671ZDEBGoximeter: registered DTrace probes
82026-03-24T22:28:56.673ZINFOoximeter: starting oximeter server file = oximeter/collector/src/lib.rs:265
92026-03-24T22:28:56.679ZDEBGoximeter: creating ClickHouse client
102026-03-24T22:28:56.757ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
112026-03-24T22:29:13.813ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:900 id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
122026-03-24T22:29:13.813ZDEBGoximeter (oximeter-agent): oximeter database does not exist, creating collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
132026-03-24T22:29:13.813ZINFOoximeter (clickhouse-client): reading db version collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:800 id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
142026-03-24T22:29:13.814ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:900 id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
152026-03-24T22:29:13.814ZINFOoximeter (clickhouse-client): read oximeter database version collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:805 id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a version = 0
162026-03-24T22:29:13.814ZINFOoximeter (clickhouse-client): initializing oximeter schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:810 id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
172026-03-24T22:29:13.814ZDEBGoximeter (clickhouse-client): initializing ClickHouse database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
182026-03-24T22:29:13.988ZINFOoximeter (clickhouse-client): inserting current version collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:831 id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a version = 14
192026-03-24T22:29:14.028ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 82d1378d-5044-44d3-9466-731b8a1deea2
202026-03-24T22:29:14.049ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /info versions = all
212026-03-24T22:29:14.049ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers versions = all
222026-03-24T22:29:14.051ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = DELETE path = /producers/{producer_id} versions = all
232026-03-24T22:29:14.051ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers/{producer_id} versions = all
242026-03-24T22:29:14.051ZINFOoximeter (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.7/src/server.rs:328 local_addr = [fd00:1122:3344:101::d]:12223
252026-03-24T22:29:14.062ZDEBGoximeter (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::d]:12223
262026-03-24T22:29:14.065ZDEBGoximeter: registered USDT probes
272026-03-24T22:29:14.065ZDEBGoximeter: contacting nexus
282026-03-24T22:29:14.128ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors
292026-03-24T22:29:14.150ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors", status: 204, headers: {"x-request-id": "8ee31d9d-0c76-4862-9989-d5707bb9e3c4", "date": "Tue, 24 Mar 2026 22:29:14 GMT"} })
302026-03-24T22:29:14.150ZINFOoximeter: oximeter registered with nexus file = oximeter/collector/src/lib.rs:428 id = d008b716-c031-4c28-af42-a025bc80f52e
312026-03-24T22:29:14.150ZINFOoximeter (oximeter-agent): starting refresh list task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:554
322026-03-24T22:29:14.151ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
332026-03-24T22:29:14.153ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?sort_by=id_ascending
342026-03-24T22:29:14.161ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "096cccb6-e2d7-48cb-8242-158bba3fbc36", "content-length": "29", "date": "Tue, 24 Mar 2026 22:29:14 GMT"} })
352026-03-24T22:29:14.161ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 0 n_pruned_tasks = 0
362026-03-24T22:29:29.151ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
372026-03-24T22:29:29.152ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?sort_by=id_ascending
382026-03-24T22:29:29.155ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "4fdf074e-2c8a-4b55-8da4-cb8869cf9277", "content-length": "721", "date": "Tue, 24 Mar 2026 22:29:29 GMT"} })
392026-03-24T22:29:29.159ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkNmE4NmEyZC04ZWNmLTRhOGItYjk3Zi1hZDRiM2MwYTNhYzkifX0%3D
402026-03-24T22:29:29.162ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkNmE4NmEyZC04ZWNmLTRhOGItYjk3Zi1hZDRiM2MwYTNhYzkifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "f75e950c-bd49-4b30-ba51-0390876b678f", "content-length": "29", "date": "Tue, 24 Mar 2026 22:29:29 GMT"} })
412026-03-24T22:29:29.162ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:8001 collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5
422026-03-24T22:29:29.162ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:4677 collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d
432026-03-24T22:29:29.162ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:45614 collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea
442026-03-24T22:29:29.162ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:59263 collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9
452026-03-24T22:29:29.162ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 4 n_pruned_tasks = 0
462026-03-24T22:29:29.163ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
472026-03-24T22:29:29.163ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
482026-03-24T22:29:29.163ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
492026-03-24T22:29:29.163ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
502026-03-24T22:29:29.184ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
512026-03-24T22:29:29.184ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
522026-03-24T22:29:29.184ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
532026-03-24T22:29:29.184ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
542026-03-24T22:29:29.184ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
552026-03-24T22:29:29.184ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
562026-03-24T22:29:29.184ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
572026-03-24T22:29:29.184ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
582026-03-24T22:29:29.194ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
592026-03-24T22:29:29.199ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 2 producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
602026-03-24T22:29:29.200ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
612026-03-24T22:29:29.218ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
622026-03-24T22:29:30.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
632026-03-24T22:29:30.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
642026-03-24T22:29:30.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
652026-03-24T22:29:30.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
662026-03-24T22:29:30.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
672026-03-24T22:29:30.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
682026-03-24T22:29:31.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
692026-03-24T22:29:31.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
702026-03-24T22:29:31.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
712026-03-24T22:29:31.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
722026-03-24T22:29:31.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
732026-03-24T22:29:31.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
742026-03-24T22:29:32.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
752026-03-24T22:29:32.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
762026-03-24T22:29:32.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
772026-03-24T22:29:32.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
782026-03-24T22:29:32.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
792026-03-24T22:29:32.167ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
802026-03-24T22:29:33.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
812026-03-24T22:29:33.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
822026-03-24T22:29:33.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
832026-03-24T22:29:33.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
842026-03-24T22:29:33.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
852026-03-24T22:29:33.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
862026-03-24T22:29:34.031ZDEBGoximeter (results-sink-cluster): inserting 380 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
872026-03-24T22:29:34.031ZDEBGoximeter (results-sink): inserting 380 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
882026-03-24T22:29:34.031ZDEBGoximeter (clickhouse-client): unrolling 380 total samples collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
892026-03-24T22:29:34.031ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
902026-03-24T22:29:34.035ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:collections"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "base_route", field_type: String, source: Metric, description: "" }, FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "producer_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "producer_ip", field_type: IpAddr, source: Metric, description: "" }, FieldSchema { name: "producer_port", field_type: U16, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.031768229Z } timeseries_name = oximeter_collector:collections
912026-03-24T22:29:34.041ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
922026-03-24T22:29:34.044ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.041272681Z } timeseries_name = ddm_router:originated_underlay_prefixes
932026-03-24T22:29:34.044ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
942026-03-24T22:29:34.045ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.044605634Z } timeseries_name = ddm_router:originated_tunnel_endpoints
952026-03-24T22:29:34.045ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
962026-03-24T22:29:34.046ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.045607765Z } timeseries_name = ddm_session:solicitations_sent
972026-03-24T22:29:34.046ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
982026-03-24T22:29:34.047ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.046535093Z } timeseries_name = ddm_session:solicitations_received
992026-03-24T22:29:34.047ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1002026-03-24T22:29:34.048ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.047506033Z } timeseries_name = ddm_session:advertisements_sent
1012026-03-24T22:29:34.048ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1022026-03-24T22:29:34.049ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.048427271Z } timeseries_name = ddm_session:advertisements_received
1032026-03-24T22:29:34.049ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1042026-03-24T22:29:34.050ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_expirations"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.049330019Z } timeseries_name = ddm_session:peer_expirations
1052026-03-24T22:29:34.050ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1062026-03-24T22:29:34.051ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_address_changes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.050230496Z } timeseries_name = ddm_session:peer_address_changes
1072026-03-24T22:29:34.051ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1082026-03-24T22:29:34.051ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_sessions_established"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.051121744Z } timeseries_name = ddm_session:peer_sessions_established
1092026-03-24T22:29:34.052ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1102026-03-24T22:29:34.052ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.052017651Z } timeseries_name = ddm_session:updates_sent
1112026-03-24T22:29:34.052ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1122026-03-24T22:29:34.053ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.052903598Z } timeseries_name = ddm_session:updates_received
1132026-03-24T22:29:34.053ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1142026-03-24T22:29:34.054ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:update_send_fail"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.053793526Z } timeseries_name = ddm_session:update_send_fail
1152026-03-24T22:29:34.054ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1162026-03-24T22:29:34.055ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.054702304Z } timeseries_name = ddm_session:imported_underlay_prefixes
1172026-03-24T22:29:34.055ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1182026-03-24T22:29:34.056ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.055603061Z } timeseries_name = ddm_session:imported_tunnel_endpoints
1192026-03-24T22:29:34.056ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1202026-03-24T22:29:34.057ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.056934232Z } timeseries_name = http_service:request_latency_histogram
1212026-03-24T22:29:34.058ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1222026-03-24T22:29:34.059ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("static_routing_config:static_routes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.058388037Z } timeseries_name = static_routing_config:static_routes
1232026-03-24T22:29:34.059ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1242026-03-24T22:29:34.060ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("static_routing_config:static_nexthops"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.059339456Z } timeseries_name = static_routing_config:static_nexthops
1252026-03-24T22:29:34.060ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1262026-03-24T22:29:34.061ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_rib:active_routes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.060308346Z } timeseries_name = switch_rib:active_routes
1272026-03-24T22:29:34.061ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1282026-03-24T22:29:34.062ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("mg_lower:routes_blocked_by_link_state"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.061225044Z } timeseries_name = mg_lower:routes_blocked_by_link_state
1292026-03-24T22:29:34.062ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1302026-03-24T22:29:34.063ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:errors_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.062143162Z } timeseries_name = switch_port_control_data_link:errors_received
1312026-03-24T22:29:34.065ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1322026-03-24T22:29:34.066ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:errors_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.065050602Z } timeseries_name = switch_port_control_data_link:errors_sent
1332026-03-24T22:29:34.066ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1342026-03-24T22:29:34.067ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:bytes_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.066279859Z } timeseries_name = switch_port_control_data_link:bytes_received
1352026-03-24T22:29:34.067ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1362026-03-24T22:29:34.068ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:packets_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.067228329Z } timeseries_name = switch_port_control_data_link:packets_received
1372026-03-24T22:29:34.068ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1382026-03-24T22:29:34.069ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:bytes_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.068189448Z } timeseries_name = switch_port_control_data_link:bytes_sent
1392026-03-24T22:29:34.069ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1402026-03-24T22:29:34.070ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:packets_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:34.069139667Z } timeseries_name = switch_port_control_data_link:packets_sent
1412026-03-24T22:29:34.082ZDEBGoximeter (clickhouse-client): inserting 26 new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1422026-03-24T22:29:34.089ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 24 table_name = fields_i16
1432026-03-24T22:29:34.091ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 8 table_name = fields_ipaddr
1442026-03-24T22:29:34.093ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 174 table_name = fields_string
1452026-03-24T22:29:34.095ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 30 table_name = fields_u16
1462026-03-24T22:29:34.096ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 24 table_name = fields_u32
1472026-03-24T22:29:34.098ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 12 table_name = fields_u8
1482026-03-24T22:29:34.100ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 90 table_name = fields_uuid
1492026-03-24T22:29:34.102ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 330 table_name = measurements_cumulativeu64
1502026-03-24T22:29:34.104ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 10 table_name = measurements_histogramu64
1512026-03-24T22:29:34.106ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 40 table_name = measurements_u64
1522026-03-24T22:29:34.135ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
1532026-03-24T22:29:34.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1542026-03-24T22:29:34.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1552026-03-24T22:29:34.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1562026-03-24T22:29:34.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1572026-03-24T22:29:34.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1582026-03-24T22:29:34.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1592026-03-24T22:29:35.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1602026-03-24T22:29:35.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1612026-03-24T22:29:35.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1622026-03-24T22:29:35.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1632026-03-24T22:29:35.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1642026-03-24T22:29:35.167ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1652026-03-24T22:29:36.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1662026-03-24T22:29:36.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1672026-03-24T22:29:36.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1682026-03-24T22:29:36.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1692026-03-24T22:29:36.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1702026-03-24T22:29:36.167ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1712026-03-24T22:29:37.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1722026-03-24T22:29:37.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1732026-03-24T22:29:37.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1742026-03-24T22:29:37.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1752026-03-24T22:29:37.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1762026-03-24T22:29:37.167ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1772026-03-24T22:29:38.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1782026-03-24T22:29:38.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1792026-03-24T22:29:38.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1802026-03-24T22:29:38.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1812026-03-24T22:29:38.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1822026-03-24T22:29:38.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1832026-03-24T22:29:39.032ZDEBGoximeter (results-sink): inserting 150 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
1842026-03-24T22:29:39.032ZDEBGoximeter (results-sink-cluster): inserting 150 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
1852026-03-24T22:29:39.032ZDEBGoximeter (clickhouse-client): unrolling 150 total samples collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
1862026-03-24T22:29:39.040ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 30 table_name = fields_string
1872026-03-24T22:29:39.041ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 36 table_name = fields_uuid
1882026-03-24T22:29:39.043ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 110 table_name = measurements_cumulativeu64
1892026-03-24T22:29:39.045ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 40 table_name = measurements_u64
1902026-03-24T22:29:39.133ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
1912026-03-24T22:29:39.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
1922026-03-24T22:29:39.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1932026-03-24T22:29:39.164ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1942026-03-24T22:29:39.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
1952026-03-24T22:29:39.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1962026-03-24T22:29:39.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1972026-03-24T22:29:39.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
1982026-03-24T22:29:39.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 2 producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
1992026-03-24T22:29:39.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2002026-03-24T22:29:40.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2012026-03-24T22:29:40.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2022026-03-24T22:29:40.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2032026-03-24T22:29:40.164ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2042026-03-24T22:29:40.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2052026-03-24T22:29:40.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2062026-03-24T22:29:41.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2072026-03-24T22:29:41.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2082026-03-24T22:29:41.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2092026-03-24T22:29:41.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2102026-03-24T22:29:41.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2112026-03-24T22:29:41.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2122026-03-24T22:29:42.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2132026-03-24T22:29:42.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2142026-03-24T22:29:42.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2152026-03-24T22:29:42.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2162026-03-24T22:29:42.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2172026-03-24T22:29:42.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2182026-03-24T22:29:43.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2192026-03-24T22:29:43.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2202026-03-24T22:29:43.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2212026-03-24T22:29:43.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2222026-03-24T22:29:43.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2232026-03-24T22:29:43.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2242026-03-24T22:29:44.031ZDEBGoximeter (results-sink): inserting 160 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
2252026-03-24T22:29:44.031ZDEBGoximeter (results-sink-cluster): inserting 160 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
2262026-03-24T22:29:44.031ZDEBGoximeter (clickhouse-client): unrolling 160 total samples collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2272026-03-24T22:29:44.041ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 50 table_name = fields_string
2282026-03-24T22:29:44.044ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 10 table_name = fields_u16
2292026-03-24T22:29:44.046ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 46 table_name = fields_uuid
2302026-03-24T22:29:44.048ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 110 table_name = measurements_cumulativeu64
2312026-03-24T22:29:44.051ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 10 table_name = measurements_histogramu64
2322026-03-24T22:29:44.053ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 40 table_name = measurements_u64
2332026-03-24T22:29:44.132ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
2342026-03-24T22:29:44.151ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
2352026-03-24T22:29:44.151ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?sort_by=id_ascending
2362026-03-24T22:29:44.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2372026-03-24T22:29:44.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2382026-03-24T22:29:44.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2392026-03-24T22:29:44.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2402026-03-24T22:29:44.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2412026-03-24T22:29:44.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2422026-03-24T22:29:44.177ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "0f4712c5-ea88-42bf-951e-9eb441832dce", "content-length": "721", "date": "Tue, 24 Mar 2026 22:29:44 GMT"} })
2432026-03-24T22:29:44.177ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::c]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkNmE4NmEyZC04ZWNmLTRhOGItYjk3Zi1hZDRiM2MwYTNhYzkifX0%3D
2442026-03-24T22:29:44.337ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::c]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkNmE4NmEyZC04ZWNmLTRhOGItYjk3Zi1hZDRiM2MwYTNhYzkifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "875f3dd5-5eb6-4874-95f8-292d1b973558", "content-length": "29", "date": "Tue, 24 Mar 2026 22:29:44 GMT"} })
2452026-03-24T22:29:44.338ZDEBGoximeter (oximeter-agent): received request to register existing metric producer, updating collection information address = [fd00:1122:3344:101::2]:32977 collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d interval = 10s producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9
2462026-03-24T22:29:44.338ZDEBGoximeter (collection-task-handle): notified collection task of new producer endpoint collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
2472026-03-24T22:29:44.338ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 4 n_pruned_tasks = 0
2482026-03-24T22:29:44.338ZDEBGoximeter (collection-task): collection task received request to update its producer information address = [fd00:1122:3344:101::2]:32977 collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d interval = 10s producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
2492026-03-24T22:29:44.339ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
2502026-03-24T22:29:44.339ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
2512026-03-24T22:29:44.378ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
2522026-03-24T22:29:45.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2532026-03-24T22:29:45.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2542026-03-24T22:29:45.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2552026-03-24T22:29:45.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2562026-03-24T22:29:45.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2572026-03-24T22:29:45.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2582026-03-24T22:29:46.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2592026-03-24T22:29:46.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2602026-03-24T22:29:46.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2612026-03-24T22:29:46.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2622026-03-24T22:29:46.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2632026-03-24T22:29:46.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2642026-03-24T22:29:47.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2652026-03-24T22:29:47.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2662026-03-24T22:29:47.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2672026-03-24T22:29:47.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2682026-03-24T22:29:47.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2692026-03-24T22:29:47.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2702026-03-24T22:29:48.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2712026-03-24T22:29:48.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2722026-03-24T22:29:48.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2732026-03-24T22:29:48.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2742026-03-24T22:29:48.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2752026-03-24T22:29:48.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2762026-03-24T22:29:49.031ZDEBGoximeter (results-sink): inserting 292 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
2772026-03-24T22:29:49.031ZDEBGoximeter (results-sink-cluster): inserting 292 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
2782026-03-24T22:29:49.031ZDEBGoximeter (clickhouse-client): unrolling 292 total samples collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2792026-03-24T22:29:49.032ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2802026-03-24T22:29:49.037ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_data_link:link_enabled"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_id", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "port_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: Bool, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.032497644Z } timeseries_name = switch_data_link:link_enabled
2812026-03-24T22:29:49.037ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2822026-03-24T22:29:49.039ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_data_link:link_up"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_id", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "port_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: Bool, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.037922501Z } timeseries_name = switch_data_link:link_up
2832026-03-24T22:29:49.039ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2842026-03-24T22:29:49.041ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:capacity"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.039869911Z } timeseries_name = switch_table:capacity
2852026-03-24T22:29:49.041ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2862026-03-24T22:29:49.043ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:occupancy"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.041650415Z } timeseries_name = switch_table:occupancy
2872026-03-24T22:29:49.043ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2882026-03-24T22:29:49.044ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:inserts"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.043219733Z } timeseries_name = switch_table:inserts
2892026-03-24T22:29:49.044ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2902026-03-24T22:29:49.046ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:deletes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.044780132Z } timeseries_name = switch_table:deletes
2912026-03-24T22:29:49.046ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2922026-03-24T22:29:49.047ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:updates"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.046286628Z } timeseries_name = switch_table:updates
2932026-03-24T22:29:49.047ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2942026-03-24T22:29:49.049ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:collisions"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.047888677Z } timeseries_name = switch_table:collisions
2952026-03-24T22:29:49.049ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2962026-03-24T22:29:49.051ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:update_misses"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.049639371Z } timeseries_name = switch_table:update_misses
2972026-03-24T22:29:49.051ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
2982026-03-24T22:29:49.052ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:delete_misses"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.051101456Z } timeseries_name = switch_table:delete_misses
2992026-03-24T22:29:49.052ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
3002026-03-24T22:29:49.053ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_table:exhaustion"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }, FieldSchema { name: "table", field_type: String, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.052616522Z } timeseries_name = switch_table:exhaustion
3012026-03-24T22:29:49.061ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
3022026-03-24T22:29:49.063ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a schema = TimeseriesSchema { timeseries_name: TimeseriesName("dendrite:sample_collection_duration"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Metric, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Metric, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Metric, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Metric, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Metric, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Metric, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Metric, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Metric, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Metric, description: "" }}, datum_type: F64, version: 1, authz_scope: Fleet, units: Count, created: 2026-03-24T22:29:49.061573517Z } timeseries_name = dendrite:sample_collection_duration
3032026-03-24T22:29:49.067ZDEBGoximeter (clickhouse-client): inserting 12 new timeseries schema collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
3042026-03-24T22:29:49.071ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 284 table_name = fields_i16
3052026-03-24T22:29:49.073ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 771 table_name = fields_string
3062026-03-24T22:29:49.075ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 142 table_name = fields_u16
3072026-03-24T22:29:49.077ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 149 table_name = fields_u32
3082026-03-24T22:29:49.079ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 148 table_name = fields_u8
3092026-03-24T22:29:49.081ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 462 table_name = fields_uuid
3102026-03-24T22:29:49.083ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 6 table_name = measurements_bool
3112026-03-24T22:29:49.084ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 215 table_name = measurements_cumulativeu64
3122026-03-24T22:29:49.086ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 1 table_name = measurements_f64
3132026-03-24T22:29:49.088ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 70 table_name = measurements_u64
3142026-03-24T22:29:49.131ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
3152026-03-24T22:29:49.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
3162026-03-24T22:29:49.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3172026-03-24T22:29:49.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
3182026-03-24T22:29:49.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3192026-03-24T22:29:49.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3202026-03-24T22:29:49.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3212026-03-24T22:29:49.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3222026-03-24T22:29:49.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 2 producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
3232026-03-24T22:29:49.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3242026-03-24T22:29:50.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3252026-03-24T22:29:50.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3262026-03-24T22:29:50.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3272026-03-24T22:29:50.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3282026-03-24T22:29:50.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3292026-03-24T22:29:50.166ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3302026-03-24T22:29:51.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3312026-03-24T22:29:51.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3322026-03-24T22:29:51.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3332026-03-24T22:29:51.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3342026-03-24T22:29:51.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3352026-03-24T22:29:51.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3362026-03-24T22:29:52.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3372026-03-24T22:29:52.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3382026-03-24T22:29:52.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3392026-03-24T22:29:52.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3402026-03-24T22:29:52.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3412026-03-24T22:29:52.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3422026-03-24T22:29:53.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3432026-03-24T22:29:53.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3442026-03-24T22:29:53.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3452026-03-24T22:29:53.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3462026-03-24T22:29:53.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3472026-03-24T22:29:53.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3482026-03-24T22:29:54.030ZDEBGoximeter (results-sink): inserting 160 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
3492026-03-24T22:29:54.030ZDEBGoximeter (results-sink-cluster): inserting 160 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
3502026-03-24T22:29:54.030ZDEBGoximeter (clickhouse-client): unrolling 160 total samples collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
3512026-03-24T22:29:54.039ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 50 table_name = fields_string
3522026-03-24T22:29:54.041ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 10 table_name = fields_u16
3532026-03-24T22:29:54.043ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 46 table_name = fields_uuid
3542026-03-24T22:29:54.045ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 110 table_name = measurements_cumulativeu64
3552026-03-24T22:29:54.047ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 10 table_name = measurements_histogramu64
3562026-03-24T22:29:54.049ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 40 table_name = measurements_u64
3572026-03-24T22:29:54.131ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
3582026-03-24T22:29:54.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3592026-03-24T22:29:54.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3602026-03-24T22:29:54.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3612026-03-24T22:29:54.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3622026-03-24T22:29:54.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3632026-03-24T22:29:54.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3642026-03-24T22:29:54.339ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
3652026-03-24T22:29:54.339ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
3662026-03-24T22:29:54.352ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = d6a86a2d-8ecf-4a8b-b97f-ad4b3c0a3ac9 producer_ip = fd00:1122:3344:101::2 producer_port = 59263
3672026-03-24T22:29:55.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3682026-03-24T22:29:55.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3692026-03-24T22:29:55.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3702026-03-24T22:29:55.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3712026-03-24T22:29:55.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3722026-03-24T22:29:55.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3732026-03-24T22:29:56.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3742026-03-24T22:29:56.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3752026-03-24T22:29:56.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3762026-03-24T22:29:56.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3772026-03-24T22:29:56.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3782026-03-24T22:29:56.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3792026-03-24T22:29:57.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3802026-03-24T22:29:57.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3812026-03-24T22:29:57.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3822026-03-24T22:29:57.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3832026-03-24T22:29:57.164ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3842026-03-24T22:29:57.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3852026-03-24T22:29:58.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3862026-03-24T22:29:58.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3872026-03-24T22:29:58.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3882026-03-24T22:29:58.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3892026-03-24T22:29:58.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
3902026-03-24T22:29:58.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
3912026-03-24T22:29:59.030ZDEBGoximeter (results-sink): inserting 292 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
3922026-03-24T22:29:59.030ZDEBGoximeter (results-sink-cluster): inserting 292 samples into database collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d
3932026-03-24T22:29:59.030ZDEBGoximeter (clickhouse-client): unrolling 292 total samples collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a
3942026-03-24T22:29:59.047ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 284 table_name = fields_i16
3952026-03-24T22:29:59.049ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 771 table_name = fields_string
3962026-03-24T22:29:59.051ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 142 table_name = fields_u16
3972026-03-24T22:29:59.053ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 149 table_name = fields_u32
3982026-03-24T22:29:59.055ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 148 table_name = fields_u8
3992026-03-24T22:29:59.057ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 462 table_name = fields_uuid
4002026-03-24T22:29:59.059ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 6 table_name = measurements_bool
4012026-03-24T22:29:59.060ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 215 table_name = measurements_cumulativeu64
4022026-03-24T22:29:59.062ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 1 table_name = measurements_f64
4032026-03-24T22:29:59.064ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d id = 248ffd8f-00d8-4228-ac39-4153ebf66d8a n_rows = 70 table_name = measurements_u64
4042026-03-24T22:29:59.131ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
4052026-03-24T22:29:59.150ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
4062026-03-24T22:29:59.150ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?sort_by=id_ascending
4072026-03-24T22:29:59.153ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "80ec9187-e8ff-4bab-aae2-22511cda2754", "content-length": "859", "date": "Tue, 24 Mar 2026 22:29:59 GMT"} })
4082026-03-24T22:29:59.153ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkNmE4NmEyZC04ZWNmLTRhOGItYjk3Zi1hZDRiM2MwYTNhYzkifX0%3D
4092026-03-24T22:29:59.155ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/d008b716-c031-4c28-af42-a025bc80f52e/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJkNmE4NmEyZC04ZWNmLTRhOGItYjk3Zi1hZDRiM2MwYTNhYzkifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "91be25f3-8790-45a6-9295-15ada4746070", "content-length": "29", "date": "Tue, 24 Mar 2026 22:29:59 GMT"} })
4102026-03-24T22:29:59.155ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::a]:37666 collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 3a78ad47-72a1-4e18-ab52-3f3e23d10cde
4112026-03-24T22:29:59.155ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 5 n_pruned_tasks = 0
4122026-03-24T22:29:59.156ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 3a78ad47-72a1-4e18-ab52-3f3e23d10cde producer_ip = fd00:1122:3344:101::a producer_port = 37666
4132026-03-24T22:29:59.156ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 3a78ad47-72a1-4e18-ab52-3f3e23d10cde producer_ip = fd00:1122:3344:101::a producer_port = 37666
4142026-03-24T22:29:59.156ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 3a78ad47-72a1-4e18-ab52-3f3e23d10cde producer_ip = fd00:1122:3344:101::a producer_port = 37666
4152026-03-24T22:29:59.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
4162026-03-24T22:29:59.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
4172026-03-24T22:29:59.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4182026-03-24T22:29:59.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4192026-03-24T22:29:59.163ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4202026-03-24T22:29:59.163ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4212026-03-24T22:29:59.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4222026-03-24T22:29:59.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 2 producer_id = 94a3b4ef-5bff-450a-9869-96732eac4bea producer_ip = fd00:1122:3344:101::2 producer_port = 45614
4232026-03-24T22:29:59.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4242026-03-24T22:29:59.168ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 5 producer_id = 3a78ad47-72a1-4e18-ab52-3f3e23d10cde producer_ip = fd00:1122:3344:101::a producer_port = 37666
4252026-03-24T22:30:00.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4262026-03-24T22:30:00.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
4272026-03-24T22:30:00.162ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4282026-03-24T22:30:00.162ZDEBGoximeter (collection-task): collecting from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4292026-03-24T22:30:00.163ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 8abb80d0-f259-485f-acb6-2adbefbd652d producer_ip = fd00:1122:3344:101::2 producer_port = 4677
4302026-03-24T22:30:00.165ZDEBGoximeter (collection-task): collected results from producer collector_id = d008b716-c031-4c28-af42-a025bc80f52e collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 27a15d3d-0dea-4ef8-88c6-53c1827bc7f5 producer_ip = fd00:1122:3344:101::1 producer_port = 8001