Max level shown:
1 [ Jan 30 21:14:08 Enabled. ]
2 [ Jan 30 21:14:08 Rereading configuration. ]
3 [ Jan 30 21:14:10 Rereading configuration. ]
4 [ Jan 30 21:14:16 Executing start method ("ctrun -l child -o noorphan,regent /opt/oxide/oximeter-collector/bin/oximeter run /var/svc/manifest/site/oximeter/config.toml --address [fd00:1122:3344:101::d]:12223 --id 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 &"). ]
5 [ Jan 30 21:14:16 Method "start" exited with status 0. ]
6 note: configured to log to "/dev/stdout"
72026-01-30T21:14:16.566ZDEBGoximeter: registered DTrace probes
82026-01-30T21:14:16.578ZINFOoximeter: starting oximeter server file = oximeter/collector/src/lib.rs:264
92026-01-30T21:14:16.584ZDEBGoximeter: creating ClickHouse client
102026-01-30T21:14:16.626ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
112026-01-30T21:14:30.201ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:900 id = 2a2534d2-2779-4600-8355-8c256551f21b
122026-01-30T21:14:30.201ZDEBGoximeter (oximeter-agent): oximeter database does not exist, creating collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d
132026-01-30T21:14:30.201ZINFOoximeter (clickhouse-client): reading db version collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:800 id = 2a2534d2-2779-4600-8355-8c256551f21b
142026-01-30T21:14:30.204ZWARNoximeter (clickhouse-client): oximeter database does not exist, or is out-of-date collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:900 id = 2a2534d2-2779-4600-8355-8c256551f21b
152026-01-30T21:14:30.204ZINFOoximeter (clickhouse-client): read oximeter database version collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:805 id = 2a2534d2-2779-4600-8355-8c256551f21b version = 0
162026-01-30T21:14:30.204ZINFOoximeter (clickhouse-client): initializing oximeter schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:810 id = 2a2534d2-2779-4600-8355-8c256551f21b
172026-01-30T21:14:30.204ZDEBGoximeter (clickhouse-client): initializing ClickHouse database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
182026-01-30T21:14:30.405ZINFOoximeter (clickhouse-client): inserting current version collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/db/src/client/mod.rs:831 id = 2a2534d2-2779-4600-8355-8c256551f21b version = 14
192026-01-30T21:14:30.440ZDEBGoximeter (clickhouse-client): registered USDT probes collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 831606a8-53b7-4278-88dc-1b8a61f93619
202026-01-30T21:14:30.472ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /info versions = all
212026-01-30T21:14:30.472ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers versions = all
222026-01-30T21:14:30.472ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = DELETE path = /producers/{producer_id} versions = all
232026-01-30T21:14:30.472ZDEBGoximeter (dropshot): registered endpoint local_addr = [fd00:1122:3344:101::d]:12223 method = GET path = /producers/{producer_id} versions = all
242026-01-30T21:14:30.473ZINFOoximeter (dropshot): listening file = /home/build/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/dropshot-0.16.6/src/server.rs:328 local_addr = [fd00:1122:3344:101::d]:12223
252026-01-30T21:14:30.486ZDEBGoximeter (dropshot): successfully registered DTrace USDT probes local_addr = [fd00:1122:3344:101::d]:12223
262026-01-30T21:14:30.486ZDEBGoximeter: registered USDT probes
272026-01-30T21:14:30.486ZDEBGoximeter: contacting nexus
282026-01-30T21:14:30.669ZDEBGoximeter: client request body = Some(Body) method = POST uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors
292026-01-30T21:14:30.699ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors", status: 204, headers: {"x-request-id": "d89cce35-1a10-40c9-b757-8e807887a651", "date": "Fri, 30 Jan 2026 21:14:30 GMT"} })
302026-01-30T21:14:30.699ZINFOoximeter: oximeter registered with nexus file = oximeter/collector/src/lib.rs:427 id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6
312026-01-30T21:14:30.701ZINFOoximeter (oximeter-agent): starting refresh list task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:554
322026-01-30T21:14:30.701ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
332026-01-30T21:14:30.705ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?sort_by=id_ascending
342026-01-30T21:14:30.715ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "259d635f-d79b-4c90-8535-fd4c587469d6", "content-length": "29", "date": "Fri, 30 Jan 2026 21:14:30 GMT"} })
352026-01-30T21:14:30.716ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 0 n_pruned_tasks = 0
362026-01-30T21:14:45.700ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
372026-01-30T21:14:45.700ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?sort_by=id_ascending
382026-01-30T21:14:45.706ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "1cf0a454-a5e6-48a2-bae6-6effd5d1a193", "content-length": "721", "date": "Fri, 30 Jan 2026 21:14:45 GMT"} })
392026-01-30T21:14:45.708ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhZGM5MWYzMS05ZDdhLTQ4MzEtYjJhMC0xZmE5OTRhMzIwYTgifX0%3D
402026-01-30T21:14:45.711ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhZGM5MWYzMS05ZDdhLTQ4MzEtYjJhMC0xZmE5OTRhMzIwYTgifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "ad75c2a3-95e7-41b3-9136-2d33f61f4dd6", "content-length": "29", "date": "Fri, 30 Jan 2026 21:14:45 GMT"} })
412026-01-30T21:14:45.711ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::1]:8001 collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326
422026-01-30T21:14:45.711ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:37579 collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 5b03a63f-53bc-4eba-a700-5c0784456ec7
432026-01-30T21:14:45.711ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:58778 collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 914343d2-5353-4fe9-86ea-d0227abefc6f
442026-01-30T21:14:45.711ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:8001 collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8
452026-01-30T21:14:45.711ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 4 n_pruned_tasks = 0
462026-01-30T21:14:45.711ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 5b03a63f-53bc-4eba-a700-5c0784456ec7 producer_ip = fd00:1122:3344:101::2 producer_port = 37579
472026-01-30T21:14:45.711ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
482026-01-30T21:14:45.712ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
492026-01-30T21:14:45.712ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 914343d2-5353-4fe9-86ea-d0227abefc6f producer_ip = fd00:1122:3344:101::2 producer_port = 58778
502026-01-30T21:14:45.752ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
512026-01-30T21:14:45.752ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 5b03a63f-53bc-4eba-a700-5c0784456ec7 producer_ip = fd00:1122:3344:101::2 producer_port = 37579
522026-01-30T21:14:45.752ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 914343d2-5353-4fe9-86ea-d0227abefc6f producer_ip = fd00:1122:3344:101::2 producer_port = 58778
532026-01-30T21:14:45.752ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
542026-01-30T21:14:45.868ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 5b03a63f-53bc-4eba-a700-5c0784456ec7 producer_ip = fd00:1122:3344:101::2 producer_port = 37579
552026-01-30T21:14:45.880ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
562026-01-30T21:14:45.894ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
572026-01-30T21:14:45.903ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 5b03a63f-53bc-4eba-a700-5c0784456ec7 producer_ip = fd00:1122:3344:101::2 producer_port = 37579
582026-01-30T21:14:45.916ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
592026-01-30T21:14:45.924ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 914343d2-5353-4fe9-86ea-d0227abefc6f producer_ip = fd00:1122:3344:101::2 producer_port = 58778
602026-01-30T21:14:45.927ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
612026-01-30T21:14:45.938ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 2 producer_id = 914343d2-5353-4fe9-86ea-d0227abefc6f producer_ip = fd00:1122:3344:101::2 producer_port = 58778
622026-01-30T21:14:46.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
632026-01-30T21:14:46.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
642026-01-30T21:14:46.713ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
652026-01-30T21:14:46.713ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
662026-01-30T21:14:46.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
672026-01-30T21:14:46.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
682026-01-30T21:14:47.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
692026-01-30T21:14:47.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
702026-01-30T21:14:47.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
712026-01-30T21:14:47.713ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
722026-01-30T21:14:47.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
732026-01-30T21:14:47.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
742026-01-30T21:14:48.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
752026-01-30T21:14:48.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
762026-01-30T21:14:48.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
772026-01-30T21:14:48.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
782026-01-30T21:14:48.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
792026-01-30T21:14:48.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
802026-01-30T21:14:49.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
812026-01-30T21:14:49.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
822026-01-30T21:14:49.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
832026-01-30T21:14:49.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
842026-01-30T21:14:49.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
852026-01-30T21:14:49.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
862026-01-30T21:14:50.444ZDEBGoximeter (results-sink-cluster): inserting 382 samples into database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d
872026-01-30T21:14:50.444ZDEBGoximeter (results-sink): inserting 382 samples into database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d
882026-01-30T21:14:50.444ZDEBGoximeter (clickhouse-client): unrolling 382 total samples collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
892026-01-30T21:14:50.444ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
902026-01-30T21:14:50.445ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("oximeter_collector:collections"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "base_route", field_type: String, source: Metric, description: "" }, FieldSchema { name: "collector_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "collector_ip", field_type: IpAddr, source: Target, description: "" }, FieldSchema { name: "collector_port", field_type: U16, source: Target, description: "" }, FieldSchema { name: "producer_id", field_type: Uuid, source: Metric, description: "" }, FieldSchema { name: "producer_ip", field_type: IpAddr, source: Metric, description: "" }, FieldSchema { name: "producer_port", field_type: U16, source: Metric, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.444752613Z } timeseries_name = oximeter_collector:collections
912026-01-30T21:14:50.451ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
922026-01-30T21:14:50.455ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.451261043Z } timeseries_name = ddm_router:originated_underlay_prefixes
932026-01-30T21:14:50.455ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
942026-01-30T21:14:50.456ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_router:originated_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.455921794Z } timeseries_name = ddm_router:originated_tunnel_endpoints
952026-01-30T21:14:50.456ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
962026-01-30T21:14:50.457ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.456884049Z } timeseries_name = ddm_session:solicitations_sent
972026-01-30T21:14:50.457ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
982026-01-30T21:14:50.458ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:solicitations_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.457843244Z } timeseries_name = ddm_session:solicitations_received
992026-01-30T21:14:50.458ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1002026-01-30T21:14:50.459ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.458755657Z } timeseries_name = ddm_session:advertisements_sent
1012026-01-30T21:14:50.459ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1022026-01-30T21:14:50.460ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:advertisements_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.459667181Z } timeseries_name = ddm_session:advertisements_received
1032026-01-30T21:14:50.460ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1042026-01-30T21:14:50.461ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_expirations"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.460568034Z } timeseries_name = ddm_session:peer_expirations
1052026-01-30T21:14:50.461ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1062026-01-30T21:14:50.462ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_address_changes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.461466698Z } timeseries_name = ddm_session:peer_address_changes
1072026-01-30T21:14:50.462ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1082026-01-30T21:14:50.463ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:peer_sessions_established"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.462378182Z } timeseries_name = ddm_session:peer_sessions_established
1092026-01-30T21:14:50.463ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1102026-01-30T21:14:50.464ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.463280465Z } timeseries_name = ddm_session:updates_sent
1112026-01-30T21:14:50.464ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1122026-01-30T21:14:50.465ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:updates_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.464194679Z } timeseries_name = ddm_session:updates_received
1132026-01-30T21:14:50.465ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1142026-01-30T21:14:50.465ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:update_send_fail"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.465113133Z } timeseries_name = ddm_session:update_send_fail
1152026-01-30T21:14:50.466ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1162026-01-30T21:14:50.466ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_underlay_prefixes"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.466055127Z } timeseries_name = ddm_session:imported_underlay_prefixes
1172026-01-30T21:14:50.466ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1182026-01-30T21:14:50.467ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("ddm_session:imported_tunnel_endpoints"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "hostname", field_type: String, source: Target, description: "" }, FieldSchema { name: "interface", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }}, datum_type: U64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.466962811Z } timeseries_name = ddm_session:imported_tunnel_endpoints
1192026-01-30T21:14:50.468ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1202026-01-30T21:14:50.469ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:errors_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.468365387Z } timeseries_name = switch_port_control_data_link:errors_received
1212026-01-30T21:14:50.471ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1222026-01-30T21:14:50.472ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:errors_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.471704594Z } timeseries_name = switch_port_control_data_link:errors_sent
1232026-01-30T21:14:50.472ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1242026-01-30T21:14:50.473ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:bytes_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.472704150Z } timeseries_name = switch_port_control_data_link:bytes_received
1252026-01-30T21:14:50.473ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1262026-01-30T21:14:50.474ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:packets_received"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.473690206Z } timeseries_name = switch_port_control_data_link:packets_received
1272026-01-30T21:14:50.474ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1282026-01-30T21:14:50.475ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:bytes_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.474659801Z } timeseries_name = switch_port_control_data_link:bytes_sent
1292026-01-30T21:14:50.475ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1302026-01-30T21:14:50.476ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("switch_port_control_data_link:packets_sent"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "asic_fab", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_lot", field_type: String, source: Target, description: "" }, FieldSchema { name: "asic_wafer", field_type: U8, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_x", field_type: I16, source: Target, description: "" }, FieldSchema { name: "asic_wafer_loc_y", field_type: I16, source: Target, description: "" }, FieldSchema { name: "kind", field_type: String, source: Target, description: "" }, FieldSchema { name: "link_name", field_type: String, source: Target, description: "" }, FieldSchema { name: "model", field_type: String, source: Target, description: "" }, FieldSchema { name: "network", field_type: String, source: Target, description: "" }, FieldSchema { name: "rack_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "sled_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "sled_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "sled_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "switch_model", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_revision", field_type: U32, source: Target, description: "" }, FieldSchema { name: "switch_serial", field_type: String, source: Target, description: "" }, FieldSchema { name: "switch_slot", field_type: U16, source: Target, description: "" }}, datum_type: CumulativeU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.475639286Z } timeseries_name = switch_port_control_data_link:packets_sent
1312026-01-30T21:14:50.484ZDEBGoximeter (clickhouse-client): retrieving timeseries schema from database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1322026-01-30T21:14:50.485ZDEBGoximeter (clickhouse-client): new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b schema = TimeseriesSchema { timeseries_name: TimeseriesName("http_service:request_latency_histogram"), description: TimeseriesDescription { target: "", metric: "" }, field_schema: {FieldSchema { name: "id", field_type: Uuid, source: Target, description: "" }, FieldSchema { name: "name", field_type: String, source: Target, description: "" }, FieldSchema { name: "operation_id", field_type: String, source: Metric, description: "" }, FieldSchema { name: "status_code", field_type: U16, source: Metric, description: "" }}, datum_type: HistogramU64, version: 1, authz_scope: Fleet, units: Count, created: 2026-01-30T21:14:50.484333872Z } timeseries_name = http_service:request_latency_histogram
1332026-01-30T21:14:50.491ZDEBGoximeter (clickhouse-client): inserting 22 new timeseries schema collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1342026-01-30T21:14:50.497ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 24 table_name = fields_i16
1352026-01-30T21:14:50.499ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 8 table_name = fields_ipaddr
1362026-01-30T21:14:50.501ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 196 table_name = fields_string
1372026-01-30T21:14:50.502ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 30 table_name = fields_u16
1382026-01-30T21:14:50.504ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 24 table_name = fields_u32
1392026-01-30T21:14:50.506ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 12 table_name = fields_u8
1402026-01-30T21:14:50.507ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 110 table_name = fields_uuid
1412026-01-30T21:14:50.509ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 322 table_name = measurements_cumulativeu64
1422026-01-30T21:14:50.513ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 10 table_name = measurements_histogramu64
1432026-01-30T21:14:50.515ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 50 table_name = measurements_u64
1442026-01-30T21:14:50.545ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
1452026-01-30T21:14:50.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1462026-01-30T21:14:50.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1472026-01-30T21:14:50.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1482026-01-30T21:14:50.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1492026-01-30T21:14:50.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1502026-01-30T21:14:50.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1512026-01-30T21:14:51.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1522026-01-30T21:14:51.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1532026-01-30T21:14:51.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1542026-01-30T21:14:51.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1552026-01-30T21:14:51.713ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1562026-01-30T21:14:51.713ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1572026-01-30T21:14:52.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1582026-01-30T21:14:52.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1592026-01-30T21:14:52.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1602026-01-30T21:14:52.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1612026-01-30T21:14:52.713ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1622026-01-30T21:14:52.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1632026-01-30T21:14:53.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1642026-01-30T21:14:53.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1652026-01-30T21:14:53.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1662026-01-30T21:14:53.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1672026-01-30T21:14:53.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1682026-01-30T21:14:53.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1692026-01-30T21:14:54.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1702026-01-30T21:14:54.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1712026-01-30T21:14:54.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1722026-01-30T21:14:54.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1732026-01-30T21:14:54.713ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1742026-01-30T21:14:54.713ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1752026-01-30T21:14:55.445ZDEBGoximeter (results-sink): inserting 200 samples into database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d
1762026-01-30T21:14:55.445ZDEBGoximeter (results-sink-cluster): inserting 200 samples into database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d
1772026-01-30T21:14:55.445ZDEBGoximeter (clickhouse-client): unrolling 200 total samples collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
1782026-01-30T21:14:55.455ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 52 table_name = fields_string
1792026-01-30T21:14:55.457ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 56 table_name = fields_uuid
1802026-01-30T21:14:55.460ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 150 table_name = measurements_cumulativeu64
1812026-01-30T21:14:55.462ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 50 table_name = measurements_u64
1822026-01-30T21:14:55.546ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
1832026-01-30T21:14:55.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1842026-01-30T21:14:55.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1852026-01-30T21:14:55.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 914343d2-5353-4fe9-86ea-d0227abefc6f producer_ip = fd00:1122:3344:101::2 producer_port = 58778
1862026-01-30T21:14:55.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1872026-01-30T21:14:55.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1882026-01-30T21:14:55.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 914343d2-5353-4fe9-86ea-d0227abefc6f producer_ip = fd00:1122:3344:101::2 producer_port = 58778
1892026-01-30T21:14:55.713ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1902026-01-30T21:14:55.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1912026-01-30T21:14:55.715ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 2 producer_id = 914343d2-5353-4fe9-86ea-d0227abefc6f producer_ip = fd00:1122:3344:101::2 producer_port = 58778
1922026-01-30T21:14:56.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1932026-01-30T21:14:56.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1942026-01-30T21:14:56.713ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1952026-01-30T21:14:56.713ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1962026-01-30T21:14:56.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1972026-01-30T21:14:56.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
1982026-01-30T21:14:57.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
1992026-01-30T21:14:57.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2002026-01-30T21:14:57.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2012026-01-30T21:14:57.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2022026-01-30T21:14:57.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2032026-01-30T21:14:57.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2042026-01-30T21:14:58.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2052026-01-30T21:14:58.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2062026-01-30T21:14:58.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2072026-01-30T21:14:58.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2082026-01-30T21:14:58.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2092026-01-30T21:14:58.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2102026-01-30T21:14:59.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2112026-01-30T21:14:59.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2122026-01-30T21:14:59.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2132026-01-30T21:14:59.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2142026-01-30T21:14:59.713ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2152026-01-30T21:14:59.713ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2162026-01-30T21:15:00.444ZDEBGoximeter (results-sink): inserting 210 samples into database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d
2172026-01-30T21:15:00.444ZDEBGoximeter (results-sink-cluster): inserting 210 samples into database collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d
2182026-01-30T21:15:00.444ZDEBGoximeter (clickhouse-client): unrolling 210 total samples collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b
2192026-01-30T21:15:00.457ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 72 table_name = fields_string
2202026-01-30T21:15:00.459ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 10 table_name = fields_u16
2212026-01-30T21:15:00.461ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 66 table_name = fields_uuid
2222026-01-30T21:15:00.463ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 150 table_name = measurements_cumulativeu64
2232026-01-30T21:15:00.466ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 10 table_name = measurements_histogramu64
2242026-01-30T21:15:00.468ZDEBGoximeter (clickhouse-client): inserted rows into table collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d id = 2a2534d2-2779-4600-8355-8c256551f21b n_rows = 50 table_name = measurements_u64
2252026-01-30T21:15:00.545ZWARNoximeter (results-sink-cluster): failed to insert some results into metric DB collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d error = Failed to check out connection to database: No backends found for this service file = oximeter/collector/src/results_sink.rs:92
2262026-01-30T21:15:00.700ZINFOoximeter (oximeter-agent): refreshing list of producers from Nexus collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:557
2272026-01-30T21:15:00.700ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?sort_by=id_ascending
2282026-01-30T21:15:00.703ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?sort_by=id_ascending", status: 200, headers: {"content-type": "application/json", "x-request-id": "473b8316-6359-42b5-a3a3-1310b4244f35", "content-length": "995", "date": "Fri, 30 Jan 2026 21:15:00 GMT"} })
2292026-01-30T21:15:00.703ZDEBGoximeter: client request body = None method = GET uri = http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhZGM5MWYzMS05ZDdhLTQ4MzEtYjJhMC0xZmE5OTRhMzIwYTgifX0%3D
2302026-01-30T21:15:00.706ZDEBGoximeter: client response result = Ok(Response { url: "http://[fd00:1122:3344:101::a]:12221/metrics/collectors/26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6/producers?page_token=eyJ2IjoidjEiLCJwYWdlX3N0YXJ0Ijp7InNvcnRfYnkiOiJpZF9hc2NlbmRpbmciLCJsYXN0X3NlZW4iOiJhZGM5MWYzMS05ZDdhLTQ4MzEtYjJhMC0xZmE5OTRhMzIwYTgifX0%3D", status: 200, headers: {"content-type": "application/json", "x-request-id": "adf6d59e-8fa1-48cb-a445-f39403c8c9fb", "content-length": "29", "date": "Fri, 30 Jan 2026 21:15:00 GMT"} })
2312026-01-30T21:15:00.706ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::a]:39406 collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5
2322026-01-30T21:15:00.706ZDEBGoximeter (oximeter-agent): registered new metric producer address = [fd00:1122:3344:101::2]:4677 collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971
2332026-01-30T21:15:00.707ZINFOoximeter (oximeter-agent): refreshed list of producers from Nexus collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d file = oximeter/collector/src/agent.rs:632 n_current_tasks = 6 n_pruned_tasks = 0
2342026-01-30T21:15:00.707ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5 producer_ip = fd00:1122:3344:101::a producer_port = 39406
2352026-01-30T21:15:00.707ZDEBGoximeter (collection-task): reporting oximeter self-collection statistics collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2362026-01-30T21:15:00.708ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5 producer_ip = fd00:1122:3344:101::a producer_port = 39406
2372026-01-30T21:15:00.708ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2382026-01-30T21:15:00.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2392026-01-30T21:15:00.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2402026-01-30T21:15:00.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2412026-01-30T21:15:00.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2422026-01-30T21:15:00.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2432026-01-30T21:15:00.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2442026-01-30T21:15:00.750ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5 producer_ip = fd00:1122:3344:101::a producer_port = 39406
2452026-01-30T21:15:00.759ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2462026-01-30T21:15:00.765ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 5 producer_id = 02d90b30-5dd7-4d08-a6dd-0c72fd2ad4a5 producer_ip = fd00:1122:3344:101::a producer_port = 39406
2472026-01-30T21:15:00.775ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2482026-01-30T21:15:01.707ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2492026-01-30T21:15:01.707ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2502026-01-30T21:15:01.708ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2512026-01-30T21:15:01.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2522026-01-30T21:15:01.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2532026-01-30T21:15:01.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2542026-01-30T21:15:01.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2552026-01-30T21:15:01.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2562026-01-30T21:15:01.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2572026-01-30T21:15:02.707ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2582026-01-30T21:15:02.707ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2592026-01-30T21:15:02.708ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 48177dbc-5a0f-4489-b1c2-3e0014e57971 producer_ip = fd00:1122:3344:101::2 producer_port = 4677
2602026-01-30T21:15:02.711ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2612026-01-30T21:15:02.711ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001
2622026-01-30T21:15:02.712ZDEBGoximeter (collection-task): sent timer-based collection request to the collection task collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2632026-01-30T21:15:02.712ZDEBGoximeter (collection-task): collecting from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2642026-01-30T21:15:02.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = adc91f31-9d7a-4831-b2a0-1fa994a320a8 producer_ip = fd00:1122:3344:101::2 producer_port = 8001
2652026-01-30T21:15:02.714ZDEBGoximeter (collection-task): collected results from producer collector_id = 26ed90ba-ff76-4ddd-aeed-e2697e7ee7d6 collector_ip = fd00:1122:3344:101::d n_results = 1 producer_id = 2ba32075-6a3a-4954-a059-7f96b96e0326 producer_ip = fd00:1122:3344:101::1 producer_port = 8001